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FOREWORD 


From FY 1993 through FY 1996, the Food and Consumer Service of the U.S. 
Department of Agriculture supported five state demonstration projects designed to test 
the feasibility and effectiveness of operating the Food Stamp Employment and Training 
(E&T) program under the same legislative and regulatory terms as the Job 
Opportunities and Basic Skills JOBS) program for AFDC recipients. Each of the 
participating states was responsible for designing and arranging for an independent 
evaluation of its demonstration. Social Policy Research Associates was selected by the 
Department of Agriculture to assist the demonstration states and their evaluation 
contractors in their evaluation efforts. As the national evaluation contractor, SPR was 
responsible for completing a critical review of each state’s evaluation design and 
providing ongoing technical assistance to the state evaluators in collecting and analyzing 
data, interpreting study findings, and preparing written evaluation reports. We were 
also charged with preparing a synthesis of state evaluation findings. 


This volume synthesizes the findings from the states’ process and implementation 
evaluations. A second volume, synthesizing the conclusions from the states’ cost and 
impact evaluations is scheduled for completion in the spring of 1997. The individual 
state-level process and evaluation reports that presented the findings summarized in this 
volume are noted as references herein. 


Over the course of the four-year demonstration period, SPR staff benefited from 
site visits to each of the demonstration states and from many telephone and written 
communications with the state evaluators and the state program staff responsible for the 
E&T/JOBS conformance demonstrations. We would like to express our appreciation 
and gratitude for the full cooperation of these individuals in making the state evaluation 
designs as consistent as possible (within the constraints established by widely varying 
demonstration designs and data sources), so that the state evaluations could support the 
national evaluation synthesis. 


We would also like to express our appreciation to Boyd Kowal, Barbara Murphy, 
and Christine Kissmer—our government technical representatives within the Food and 
Consumer Service’s Office of Analysis and Evaluation—and to Micheal Atwell of the 
Food and Consumer Service’s Program Development Division for their support and 
encouragement over the course of the project. 
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EXECUTIVE SUMMARY 


From October 1, 1993 through September 30, 1996, the Food and Consumer 
Service of the U.S. Department of Agriculture supported demonstration projects in five 
states—Georgia, Hawaii, Missouri, South Dakota, and Texas—to test the feasibility and 
effectiveness of operating the Food Stamp Employment and Training (E&T) program 
under the same legislative and regulatory terms as the Job Opportunities and Basic 
Skills (JOBS) program for AFDC recipients. Common objectives of the 
demonstrations included: (1) increasing administrative efficiency through the 
simplification of work registration and sanctioning procedures across welfare-to-work 
programs; (2) increasing the range and intensity of education, vocational training, and 
job search services available through the E&T program; (3) providing services that are 
well-matched to individual needs; and (4) improving client outcomes. This Synthesis of 
Implementation and Process Evaluations summarizes the findings from the process and 
implementation evaluations prepared by the demonstration states and their local 
independent evaluators. 


To conform their E&T programs to the JOBS model, states made a variety of 
changes to participation rules, including requiring mandatory work registration by 
members of new groups (primarily caretakers responsible for children between 3 and 6 
years of age and recipients of unemployment benefits), giving priority to members of 
groups expected to be at risk of long-term dependency, implementing tougher 
sanctioning policies, and encouraging participation by volunteers. To conform their 
E&T service designs to the JOBS model, states undertook to: provide assessment and 
individual service planning to participants; increase the range of education and training 
services available to participants through non-reimbursable coordination linkages 
and/or direct purchase of enhanced services; require participation in educational 
components by enrollees who had not completed high school; and offer more generous 
payment schedules for reimbursement of transportation, child care, and other expenses 
associated with participation in education and training activities. 


KEY FINDINGS FROM THE IMPLEMENTATION EVALUATIONS 

The implementation experiences of the five demonstration states indicate that 
conforming the rules and regulations of the E&T and JOBS programs is feasible. 
Moreover, in each demonstration state, conformance of the work registration, 
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exemption, and sanctioning rules and procedures between E&T and JOBS was 
perceived as a significant improvement over the operation of two different welfare-to- 
work programs with different rules and regulations. In most states, demonstration 
operations were perceived by staff as providing more meaningful services to 
participants who were more motivated than under the previous E&T’ programs. 


However, the demonstration programs were substantially more expensive to 
operate than the E&T programs they replaced, even when most of the actual cost of 
training was leveraged from other funding streams.! As a result, even with the 
cooperative agreement funds, each of the demonstrations found that it could serve 
fewer participants than previously. Selective recruitment and enrollment were key to 
the ability of most of the demonstrations to stay within their budgets. In addition, 
states that tested conformance on a large geographic scale sometimes placed limits on 
the services available to participants (particularly supportive services) to constrain 


demonstration costs. 


In preparing for the start-up of demonstration services, states were able to adapt 
JOBS policies and procedures as well as management information and reporting 
systems developed for the JOBS program. This strategy generally worked well, except 
in several instances in which the JOBS automated information system was undergoing 
redesign or was not yet operational at the time of demonstration start-up. Although 
demonstration planning and start-up went smoothly in a number of states, developing 
new service delivery arrangements was a very time consuming process in several states 
that created new delivery systems for demonstration services. In addition, staff 
turnover created operational challenges in several states after the demonstrations were 
underway. Overall, program operations appear to have achieved the intended 
demonstration service designs. 


Perhaps the most striking operational issue identified by the demonstration states 
was difficulty identifying and recruiting appropriate participants. In all the 
demonstration states, a significant proportion of the referred work registrants were 
already off Food Stamps or exempt by the time they were called in for services. Asa 
result, all the demonstrations had to call in substantially larger numbers of work 
registrants than they had the capacity to enroll. 


1The impact of the conformance demonstrations on the cost of E&T services will be described in 
more detail in the Synthesis of Cost and Impact Evaluations. 
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Among the implementation lessons suggested by the experiences of the 
demonstration states are the following: 


e Involve all affected agencies and levels in planning for demonstration 
services. 


e Cross-train staff, including income maintenance workers, to understand 
how their actions affect the rest of the system. 


e Take advantage of non-financial coordination linkages to increase access 
by E&T participants to education and training services. 


e Do not expect a tougher sanctioning policy, by itself, to increase rates 
of participation or decrease sanctioning rates. 


e Recognize that the availability of child care expense reimbursements 
may be a powerful magnet encouraging participation by exempt work 
registrants (“exempt volunteers”) and mandatory work registrants who 
may volunteer to participate prior to being called in (sometimes referred 
to as “mandatory volunteers”). 


KEY FINDINGS FROM THE PROCESS EVALUATIONS 

Although the five states participating in the E&T/JOBS conformance 
demonstration shared overall project goals and used the JOBS model as the framework 
for redesigning their E&T programs, there were significant variations across states in 
demonstration policies and practices. Exhibit A summarizes some of the key variations 
across the five demonstration states in the services, operational arrangements, and 
targeting/participation designs implemented as part of the conformance demonstrations. 


Individual assessment and service planning provided by case management staff 
were key elements of the demonstration designs in all states. As a result, the number 
of work registrants participating in each demonstration was constrained by the ability of 
case managers to schedule individual intake sessions and manage their assigned 
caseloads. Despite variations in client targeting policies and call-in procedures, each 
demonstration enrolled only a relatively small proportion, ranging from 1 percent to 10 
percent, of the mandatory work registrant pool. 


Four of the five demonstration states developed policies of giving priority to 
“high risk” target groups. Members of target groups made up over half of the 
demonstration participants in each state, but most states also served non-target group 
members. Four of the five states also gave priority to participation by volunteers. As 
a result of the active recruitment of volunteers, volunteers made up more than 60 
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EXHIBIT A 


KEY FEATURES OF THE FOOD STAMP E&T/JOBS 


Georgia’s PEACH 
JET Demonstration 


Hawaii’s PRIDE 
Demonstration 


Missouri’s JET 
Demonstration 


South Dakota’s 
E&T/JOBS 
Conformance 
Demonstration 


Texas’ BOND 
Demonstration 


Official Demonstration 
Name 


JOBS Employment and Training 
Program (JET) under the umbrella 
of the state’s Positive Employment 
and Community Help (PEACH) 
Program 


Positive Response in Developing 
Employment (PRIDE) 


JOBS-Employment and Training 
Demonstration (JET) 


Family Independence Food Stamp 
Employment and Training/JOBS 
Conformance Demonstration 


Better Opportunities for New 
Directions (BOND) 
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CONFORMANCE DEMONSTRATIONS 


Key Service Features 
(Compared to Non- 
Demonstration Sites/Periods) 


Expanded supportive services 


Often more intensive assessment 
and case management than 
comparison sites 


Some training purchased directly 
on behalf of demonstration 
participants (e.g., on-site adult 
education classes) 


Health and psycho-social 
assessments 


Efforts by social worker to link 
participants to services needed to 
overcome employment barriers 


Increased referrals to education 
and training 


Expanded supportive services 


Individualized service planning and 
case management 


Access to demonstration-funded 
education and training 


Individual orientation and service 
planning session 


Availability of a broader range of 
services, including referrals to 
education and training 


Availability of service sequences 
appropriate for job-ready and less 
job-ready clients 


Case management services 


Increased referrals to education 
and training 


Availability of expanded 
supportive services for all 
participants 





Georgia’s PEACH 
JET Demonstration 


Hawaii’s PRIDE 
Demonstration 


Missouri’s JET 
Demonstration 


South Dakota’s 
E&T/JOBS 
Conformance 
Demonstration 


Texas’ BOND 
Demonstration 


EXHIBIT A 
KEY FEATURES OF THE FOOD STAMP E&T/JOBS 


CONFORMANCE DEMONSTRATIONS, 


CONTINUED 


Scope of 
Demonstration 


Implemented in four 
counties with about 16% 
of state’s Food Stamp 


population 


Implemented in one 
county with about 70% 
of state Food Stamp 
cases 


Implemented in eight 
counties with about 16% 
of statewide E&T work 
registrant pool 


Implemented in all 20 
counties that have 
operating E&T programs 


Implemented in one 
county with less than 2% 
of all state E&T work 
registrants 
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Changes in Non- 
Demonstration 
Services 


Statewide E&T program 
also adopted many 
aspects of JOBS service 
design simultaneously 
with demonstration, 
including use of JOBS 
service components 


Statewide E&T program 
unchanged 


Statewide E&T program 
changed somewhat to 
offer same service 
components as JOBS 


There was no residual 
non-demonstration E&T 
program 


Statewide E&T services 
for job-ready clients also 
evolved closer to the 
JOBS design 


Operational Features 


Extent of 
Consolidation with 
JOBS 


E&T operations 
consolidated with JOBS 
throughout state, but 
separate case managers 
for E&T caseloads 


E&T demonstration 
operated separately from 
JOBS 


E&T demonstration 
operated separately from 
JOBS, but staff trained 
and supervised by JOBS 
unit 


Single program 
operations handbook for 
E&T and JOBS, but 
operation of two 
programs is not 
consolidated 


E&T and JOBS 
programs consolidated in 
demonstration site, 
including the 
administration of 
integrated E&T/JOBS 
caseloads by BOND case 
managers 





EXHIBIT A 
KEY FEATURES OF THE FOOD STAMP E&T/JOBS 
CONFORMANCE DEMONSTRATIONS, 
CONTINUED 


Design for Client Targeting and Participation 





















Policy on Client Targeting Limits to Service 
Participation by Policies Capacity 
Volunteers 






















Services targeted to 
priority groups modeled 
after JOBS target groups 


Participation by 
volunteers encouraged; 
many volunteers have 
applied for child care 
services to support 
training and employment 


Georgia’s PEACH 
JET Demonstration 






















Services targeted to 
priority groups including 
some JOBS target 
groups and some local 
target groups 


Participation limited by 
small caseload size 


Participation by 
mandatory and exempt 
volunteers particularly 
encouraged 


Hawaii’s PRIDE 
Demonstration 





























Missouri’s JET Participation by Services targeted to Participation limited by 
Demonstration volunteers particularly priority groups modeled | small caseload size 
encouraged after JOBS target groups 


















Priority groups similar 
to JOBS; state would 
like to call-in all 
mandatory work 
registrants 


Participation by 
volunteers not 
encouraged 


South Dakota’s 
E&T/JOBS 
Conformance 
Demonstration 



































No priority groups 
established; state targets 
services to all expected 
to benefit from 
participation 


Participation in intensive 
services limited by small 
caseload size 


Participation by 
volunteers encouraged 


Texas’ BOND 
Demonstration 
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percent of the demonstration participants in two states and between 30 percent and 40 
percent of all participants in the two additional states that encouraged volunteers. 


Although it was hoped that the tougher sanctioning policies associated with JOBS 
would increase participation rates under the demonstrations, not all states achieved this 
result. In general, the states with lower call-in rates and higher proportions of 
voluntary participants issued notices of adverse action at a lower rate under the 
demonstration than previously, while the states that called in high proportions of the 
mandatory work registrant pool continued to experience high “no show” rates. In 
comparison with previous E&T services, demonstration service designs were 
distinguished by (1) redesigned and intensified job training and job search components 
that responded to participants’ need for improved self-confidence and motivation; (2) 
greater flexibility of service sequences and improved access to education and vocational 
training opportunities; (3) more thorough assessment, individualized service planning 
and intensified case management; and (4) availability of enhanced supportive services. 
All the demonstration states appear to have been successful in increasing the range of 
training options available to and used by participants. In several states, regular E&T 
programs were also evolving along similar lines, but, in practice, the demonstration 
efforts in most states represented a qualitative step forward, away from the one-size- 
fits-all approach and towards individualized services. 


The implementation and process evaluation reports document increased 
participation in education and vocational training by E&T participants as the result of 
the demonstrations’ individualized service planning processes. The proportion of E&T 
participants enrolled in educational activities—including adult education, post- 
secondary education, and high school—increased under the demonstration in most 
states. Thus, the JOBS program’s objective of promoting educational development, 
particularly for young people without a high school education, appeared to have been 
realized in the demonstrations. However, states appeared to be less successful in 
boosting opportunities for vocational skills training and on-the-job training. 


The shift towards more intensive education and training services was 
accomplished primarily by linking participants to existing education and training 
resources in the local community, such as adult education programs, training offered 
by local post-secondary schools, and JTPA-funded training. Aspects of the 
demonstration designs that were associated with the increased utilization of education 
and training services by demonstration participants included: (1) providing of 
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individualized assessment and counseling by case managers who encouraged 
participants to pursue education or training when needed; (2) giving priority to 
participation by volunteers who were already motivated to pursue education or training; 
(3) requiring individuals under 24 without a high school diploma to participate in 
education; (4) developing strong non-reimbursable and/or funded referral arrangements 
between the demonstration and existing community education and training resources; 
and (5) making available enhanced supportive services that could help support 
participants while they were in training. 


The five demonstration states illustrate several different approaches to allocating 
available resources among potential program participants and across different program 
activities. Texas and Georgia each designed multiple service tracks that offered 
services to both job-ready and less job-ready work registrants. Both states decided that 
it would be desirable to enroll mandatory and exempt volunteers in self-initiated 
training and support them during training by offering them case management and 
supportive services. Both states also called in significant numbers of mandatory work 
registrants for individual job search counseling (Georgia) or group job readiness/job 
search workshops (Texas). 


In contrast to the multiple track systems in Texas and Georgia, Missouri and 
South Dakota each utilized a single service approach. Missouri’s demonstration design 
was targeted primarily to hard-to-serve exempt and mandatory volunteers who were 
interested in pursuing further education and training. A very small percentage of the 
mandatory work registrant caseload was served in this state. South Dakota’s design, 
on the other hand, was targeted almost exclusively to mandatory work registrants. 
While the menu of services was flexible enough to support participation in education 
and training when education was identified as part of a participant’s employment plans, 
South Dakota’s program was primarily oriented to encourage immediate employment 
among job-ready clients. 


Hawaii was distinctive among the demonstration states because of its broad 
family-focused and multi-disciplinary service approach . Like Missouri, Hawaii 
attempted to reach out to the least job-ready individuals among the work registrant 
pool. However, Hawaii attempted to prepare mandatory work registrants for 
successful labor market participation by providing referrals to individualized front-end 
multi-disciplinary assessment of employment barriers, community health and 
counseling resources, and intensive case management services. The community 
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resources Hawaii tried to leverage on behalf of its participants included not only 
education and vocational training resources, but health, mental health, and family 
Support services. After addressing some of these personal employment barriers, 
participants in Hawaii’s E&T demonstration were referred to a combination of job 
readiness and job search support services supplemented by basic education when 
needed. 


CONCLUSIONS 

Assessments of whether the services provided to participants under the 
conformance demonstrations led to improved outcomes are currently being performed 
by the states in their impact evaluations and will be reported in SPR’s Synthesis of Cost 
and Impact Evaluations. Without information about how the demonstrations affected 
participant outcomes, it is difficult to determine whether the demonstrations 
accomplished their objectives. 


The findings from the states’ process and implementation evaluations have set the 
stage for these later studies by describing the implementation experience and detailing 
differences in client flows and service strategies in demonstration and comparison 
counties within each state. Because of the unique configuration of each demonstration 
and its non-demonstration context, the impact evaluation findings will not lend 
themselves to easy comparisons across the demonstration states. In interpreting cross- 
state differences in impact and cost findings, it will be important to keep in mind the 
exact differences between the demonstration and non-demonstration interventions in 
each particular state. For example, impact evaluations in Hawaii and South Dakota are 
comparing the demonstrations to “old style” E&T operations in a non-demonstration 
site (Hawaii) or during a pre-demonstration period (South Dakota), whereas the impact 
evaluations in Georgia, Missouri, and Texas are comparing the demonstrations to 
transformed E&T programs in non-demonstration sites that had simultaneously adopted 
a number of the same JOBS program features that characterize the demonstrations. 


The experiences of the states participating in the E&T/JOBS Conformance 
Demonstration have to be reevaluated in terms of their applicability to the new world 
of welfare-to-work that was finally enacted in August 1996 by the Personal 
Responsibility and Work Opportunities Reconciliation Act of 1996 (PRWORA). 
PRWORA replaced AFDC and JOBS with state block grants that provide states with 
considerable design flexibility but within a context of stringent work participation 
requirements that not only emphasize “work first” but also place restrictions on how 
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participation in education will be counted. PRWORA also places a maximum of five 
years on the receipt of cash assistance under the new program of Temporary Assistance 
to Needy Families (TANF) and limits the Food Stamp eligibility of able-bodied adults 
between the ages of 18 and 50 who do not have dependent children to three months out 
of a three year period, unless they are employed 20 hours a week. In this context, 
states will be under pressure to design services that place large numbers of TANF 
recipients in work activities while reserving education and vocational training for 
limited subgroups, such as teen parent head of households and individuals who 
simultaneously meet the requirement for 20 hours per week of work participation. 


It is not yet clear how the new TANF work requirements and the Food Stamp 
program eligibility limits will affect various states’ decisions on target group strategies 
and service designs for the E&T program. Possible E&T target groups relevant to the 
new PRWORA context include individuals without dependent children who are at risk 
of losing Food Stamp eligibility after three months unless they work at least 20 hours a 
week, individuals who have recently lost eligibility for cash grants under TANF due to 
the expiration of their time limit, as well as individuals at risk of long-term Food Stamp 
recipiency. In developing E&T service approaches, states will have to choose between 
an E&T program that conforms with the primary thrust of their Title [V-A welfare-to- 
work program—i.e., a program that encourages widespread participation in services 
oriented to immediate employment—and an E&T program that offers a variety of 
services selectively targeted and/or individually matched to meet the widely differing 
circumstances of different subgroups receiving assistance from the Food Stamp 
program. 


On the one hand, the experience of the states participating in the E&T/JOBS 
Conformance Demonstration suggests that state E&T service designs that attempt to 
conform E&T and TANF requirements—where permissible under Department of 
Agriculture regulations for the E&T program—will be administratively feasible. On 
the other hand, while conformance with TANF work requirements and service designs 
may simplify the administration of welfare-to-work systems, states would also do well 
to consider that a number of the conformance demonstration states were able to offer 
more intensive services to E&T participants who were motivated to improve their job- 
related skills through education or training. 


The process evaluations for the E&T/JOBS Conformance Demonstration illustrate 
how the demonstration states addressed E&T client targeting and program design 
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choices that now face all states in the context of welfare reform. As the demonstrations 
in Texas, Georgia, and Hawaii showed, it is feasible to design programs with multiple 
service tracks that provide large numbers of job-ready clients with services oriented to 
immediate employment as well as smaller numbers of “hard-to-employ” individuals 
with services designed to support participation in education and training. Key among 
the new services offered under the E&T/JOBS Conformance Demonstrations were 
supportive services, which, in combination with well-developed coordination linkages, 
enabled participants to attend education or training services available through existing 
community resources. 


As the only remaining form of welfare-to-work programming that offers an 
uncapped 50 percent match of State expenditures, the E&T program could become an 
increasingly important source of funds to states that are interested in improving the 
employability of hard-to-serve individuals who are ineligible for federally-assisted cash 
assistance to needy families. States’ interests in investing in the future of these 
households may increase over time, as time limits begin impacting TANF eligibility for 
increasing numbers of poor Americans. 
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I CONFORMANCE BETWEEN E&T AND JOBS: AN 
OVERVIEW 


HISTORY AND OVERVIEW OF THE DEMONSTRATION 


The Food Stamp Employment and Training (E&T) program, initiated by the Food 
Stamp Act of 1985 and required to be in operation in all states by April 1, 1987, was 
intended by Congress to increase the employability of program participants by 
“[assisting] members of households participating in the Food Stamp program in gaining 
skills, training, or experience that will increase their ability to obtain regular 
employment.” However, the program has been allotted only limited federal funding—it 
expends less than $140 million annually—to realize these lofty goals.! When 
distributed across the 1.3 million individuals who participate in E&T services ina 
typical year, the program spends only about $100 in federal funds, on average, per 
participant. As a result of cost-sharing requirements, the states contribute additional 
funds, accounting for another $60 million in expenditures annually, which increases the 
total expended to about $200 million nationally, or about $150 per participant. 


Previous E&T program designs have resulted in large numbers of work 
registrants receiving minimal services as well as the issuance of large numbers of 
sanctions for noncompliance. 2 Prior to FY 1992, E&T program design and client 
targeting decisions were strongly influenced by federal performance standards requiring 
states to serve at least 50% of all mandatory nonexempt work registrants either by 
enrolling them in component services or sanctioning them for failure to comply with 
participation requirements. In response to these federal requirements, most states 
targeted E&T services broadly to all mandatory work registrants but offered only 
limited services, primarily job search training and individual job search assistance. 
Furthermore, because sanctioning procedures had no “teeth,” E&T case managers 


1 The Department of Agriculture provides $75 million annually to the states as 100% federally- 
funded formula grants for the administration and operation of E&T services. Additional federal funds 
are available on a dollar-for-dollar matching basis to states that want to spend more on E&T operations. 
The federal government also provides matching funds to states to reimburse participants for the costs of 
child care and transportation/training expenses, within federal cost limits. Supportive service 
expenditures in excess of these limits must be provided using nonfederal funds. 


2 Social Policy Research Associates and SRI International, Study of the Food Stamp Employment 
and Training Program: Operations, Funding, and Coordination. May 1992. 
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spent much of their time issuing and curing sanctions, rather than helping participants 
find jobs. 3 Perhaps because of the limited services provided to most participants, a net 
impact study performed during the first year of program operations found that the 
program had failed to achieve any statistically significant improvements in employment 
outcomes for E&T participants, compared to what they would have achieved without 
the program.‘ 


Thus, at the end of FY 1991, state administrators of the Food Stamp Employment 
and Training Program faced an important turning point. They were eager for ideas 
about how to transform the program from one widely perceived as merely an 
administrative requirement for Food Stamp recipients and a paper-processing nightmare 
for program staff into a program providing meaningful and effective employment 
services. Responding to criticisms of the previous E&T program design, the 
Department of Agriculture opened the door to program redesigns at the state level by 
reducing the required participation rate standard from 50% to 10% of mandatory work 
registrants, effective FY 1992. 


The Job Opportunities and Basic Skills (JOBS) program for AFDC recipients 
offered a potentially attractive model to states interested in developing new approaches 
for the E&T program. At the time that E&T was facing a turning point, the JOBS 
program was being touted as the answer to improving the employability of AFDC 
recipients. The JOBS program design 


e Utilized selective targeting to emphasize serving to clients who would 
otherwise be at risk of long-term welfare dependency. 


e Emphasized individualized service planning, rather than a “one size fits 
all” approach. 


e Emphasized the improvement of participant employability through the 
delivery of basic education and vocational training services. 


e Built on coordination linkages with basic education and vocational 
training providers to leverage additional public funds on behalf of 
program participants. 


3 Participants could cure a sanction merely by indicating their willingness to cooperate with the 
program, without taking meaningful steps to comply. 


4 Abt Associates, Inc., Evaluation of the Food Stamp Employment and Training Program Final 
Report. June 1990. 


In recognition of the need to test new models for E&T design and operations, 
The Hunger Relief Act of 1991 authorized the Secretary of Agriculture to conduct a 
demonstration on “conforming the Food Stamp E&T and JOBS programs in 60 project 
areas.” On March 27, 1992, the Department of Agriculture issued an announcement 
inviting states to submit proposals for operating their E&T program in selected project 
areas under the same legislative and regulatory terms as the JOBS program.>* States 
were also encouraged to develop partnerships among different employment and training 
programs to achieve greater coordination between E&T and other programs like the 
Job Training Partnership Act (JTPA), adult basic education, and vocational education. 


To conform E&T participation rules and procedures to JOBS policies and 
practices, states interested in participating in the demonstration were permitted to 
change E&T operations in the following ways: 


e Replace E&T exemption criteria with JOBS exemption criteria. Key 
differences were the inclusion in JOBS mandatory work registration 
rules of individuals responsible for the care of dependent children over 
3 years of age (over 1 year of age at state option), rather than over 6 
years of age as in E&T requirements, and the inclusion of individuals 
receiving UI benefits. 


e Give priority to participation by volunteers, subject to resource 
availability. The JOBS legislation called for priority both to exempt 
work registrants and mandatory work registrants who volunteered prior 
to being called in for services. One group of volunteers cited in the 
legislation consisted of mandatory or exempt volunteers already enrolled 
in self-initiated training when they entered the program. Individuals in 
self-initiated training approved by JOBS were eligible to receive 
supportive services from the JOBS program. 


e Implement selective targeting of clients, following the state’s JOBS 
example. For the AFDC population served by JOBS, these target 
groups included individuals under age 24 without a high school diploma 
or GED, individuals under age 24 with little or no work experience, 
individuals who had received AFDC benefits for 36 or more of the 60 
months prior to certification, and members of households who were 
scheduled to lose their AFDC eligibility within two years because their 
dependent children would “age out” of the program. Under JOBS, 


5Although the Department of Agriculture encouraged states to propose demonstrations that tested 
full conformance between the E&T and JOBS programs, states were permitted to submit proposals for 
limited conformity, if they cited barriers that prevented them from guaranteeing full conformance. 
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participants in target groups had to account for at least 55% of all 
program expenditures. 


e Replace E&T sanctioning rules with JOBS sanctioning rules. These 
rules were generally perceived as being more stringent than E&T 
sanctioning policies because they had more serious consequences for the 
second and third occurrences of noncompliance. However, under 
JOBS, sanctions applied only to the noncompliant individual, rather 
than to the entire household.°® 


To achieve conformance of E&T service designs with JOBS, states were 


encouraged to: 


e Provide assessment and individual service planning to all 
participants. In JOBS, assessment and individual service planning was 
required for all participants. Under JOBS, states could define 
assessment as a service component for the purposes of computing client 
participation hours during the first month of J OBS participation. 


e Offer the same service components available to JOBS participants 
and require clients to participate in at least 24 hours of E&T 
activities per week. JOBS service components varied by state but had 
to include education, job skills training, job readiness activities, and job 
development and placement assistance, as well as two of the following 
four optional service components: group or individual job search, on- 
the-job training, work supplementation, and community work 
experience or another approved work experience program. Case 
management was also a permitted JOBS service component, at 
individual state option. 


e Develop non-reimbursable coordination agreements for the delivery 
of a wide range of education and training services to participants 
through individual referral arrangements. To provide more intensive 
education and training services, the JOBS legislation encouraged 
programs to develop non-financial coordination agreements with a range 
of local education and training providers as a supplement to services 
provided directly with JOBS funds. 


e Use JOBS procedures to match clients to individual or sequenced 
services. In contrast to E&T service designs, which often required all 
enrollees to participate in job search or job search training as the first 
component, JOBS service assignment procedures often were based on 


6 The JOBS sanctioning rules required participants to complete a 3-month and 6-month sanction 
period for second and third sanctions, respectively, before they could request to have their benefits 
reinstated. 
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an individual assessment of participant needs and participation in 
multiple services to address identified needs. 


e Require certain participants to enroll in education activities, if these 
participants had not completed high school. Under JOBS, mandatory 
work registrants who were caretakers under 24 years of age were 
required to participate in GED programs if they had not completed high 
school. Caretakers under 20 years of age had to participate in education 
regardless of the ages of their children. 


e Offer supportive services consistent with the state’s JOBS supportive 
service plan. JOBS programs usually offered a higher level of 
reimbursement for a broader range of work-related expenses compared 
to the $25 per month transportation reimbursement available under E&T 
(JOBS offered reimbursements for books, uniforms, tuition, automobile 
repair, and dental work). JOBS also offered more generous child care 
expense reimbursements. 


Each of these program features offered an alternative to the then-common E&T 
program design of universal targeting with a standardized service sequence that was 
perceived as not intensive enough to make a difference for most participants. 
Implementing uniform work registration and sanctioning procedures between E&T and 
JOBS was attractive because it would enable states to simplify and consolidate time- 
consuming and complex administrative features of the E&T and JOBS programs. In 
addition, it was hoped that using JOBS’ tougher sanctioning procedures for mandatory 
work registrants in E&T would reduce the rate of noncompliance and free up staff time 
spent tracking participation and requesting sanctions. Staff could then spend more time 
providing employment and training services to program participants. States with low 
AFDC benefit levels—where many JOBS participants became ineligible for AFDC 
benefits as soon as they obtained a minimum wage job—also were eager to facilitate the 
transfer of individual participants from JOBS to E&T, and vice versa, without 
interrupting the delivery of employment and training services, as participants moved 
between public assistance (PA) and Food Stamps/non-PA status. 


In replicating the JOBS service model for E&T participants, states also hoped 
that they would be able to build on the coordination networks developed by JOBS to 
leverage funds from other programs—such as the JTPA and adult education systems— 
to pay for the education and vocational training services received by program 
participants. In addition, the possibility of service consolidation for E&T and JOBS 
offered the potential to realize cost savings in the delivery of services for both JOBS 
and E&T through economies of scale. Although they were recognized to be potentially 


expensive, the enhanced supportive services available under JOBS were perceived by 
some states to be key to increasing participant access to more intensive education and 


training services. 


Five states were ultimately selected for participation in the Food Stamp 
E&T/JOBS Conformance Demonstration.’? The approved demonstration projects 
included the following: 

e Georgia’s JOBS Employment and Training (JET) Program operated 


under the umbrella of the state’s Positive Employment and Community 
Help (PEACH) Program. 


e Hawaii’s Positive Response in Developing Employment (PRIDE) 
Program. 
e Missouri’s JOBS-Employment and Training (JET) Demonstration. 


e South Dakota’s Family Independence Food Stamp Employment and 
Training/JOBS Conformance Demonstration. 


e Texas’ Better Opportunities for New Directions (BOND) Program. 


Although the official demonstration period was a full four years—from October 
1, 1992 through September 30, 1996—the participating states used the first six to 18 
months after the announcement of demonstration funding to prepare for project start- 
up. Objectives of the demonstration common to the demonstration states of Georgia, 
Hawaii, Missouri, South Dakota, and Texas included: (1) increasing administrative 
efficiency through the simplification of work registration and sanctioning procedures 
across welfare-to-work programs; (2) increasing the range and intensity of education, 
vocational training, and job search services available through the E&T program; (3) 
providing services that were well-matched to the needs of job-ready and less job-ready 
individuals through the use of assessment and individualized service planning; and (4) 
improving client outcomes.® 


7 The official demonstration title was the E&T/JOBS/JTPA Conformance Demonstration. 
Inclusion of the Job Training Partnership Act in the official title reflected the federal government’s 
interest in improving non-financial coordination linkages with JTPA-funded programs to support the 
provision of individualized education and training services to demonstration participants. Strong linkages 
with the JTPA system did not emerge, however, as central features of the approved demonstrations. 


8 See the Appendices for a description of the specific demonstration objectives identified by each 
state. 











Under a cooperative agreement with each demonstration state, the Department of 
Agriculture authorized the conformance demonstrations in designated project areas and 
provided each state between $500,000 and $600,000 to cover the increased costs of 
demonstration activities, including the increased costs of enhanced supportive services 
for demonstration participants. The cooperative agreements also specified that each 
state should select an independent evaluator and conduct an evaluation of the 
demonstration. 


Within the common framework established by the demonstration guidelines and 
shared state objectives, the demonstration states varied substantially in: 


e The number of local E&T sites (and percentage of all statewide work 
registrants) involved in the demonstration. 


e How priority target groups were defined, how potential participants 
were selected or recruited for services, and the extent that participation 
by volunteers was attempted and/or achieved. 


e The particular service components available to demonstration 
participants, and how participants were matched to services. 


e Whether the demonstration involved actual consolidation of E&T 
operations with JOBS or the operation of separate but parallel 
programs. 


The demonstration states also varied in whether they transformed their statewide 
E&T program at about the same time as, but independently of, the demonstration, or 
whether they retained the “old style” broadly-targeted services in non-demonstration 
counties. To some extent, states could model their E&T programs after the JOBS 
model without receiving waivers from the Department of Agriculture. Under the E&T 
program rules in effect starting in FY 1992, states could adopt JOBS service 
components, implement selective client targeting policies similar to the JOBS target 
groups, and consolidate the delivery of E&T and JOBS services through integrated 
service contracts or the use of integrated in-house employment and training units. In 
fact, three of the five states that were selected for participation in the E&T/JOBS 
Conformance Demonstration (Missouri, Texas, and Georgia) adopted client targeting 
and service offerings modeled after their JOBS programs for their E&T programs on a 
statewide basis. However, without an official demonstration waiver, states could not 
use JOBS work registration or sanctioning criteria or receive federal support for the 
cost of enhanced supportive services for E&T participants. 
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OVERVIEW OF THE EVALUATION 

To document the key similarities and differences among the demonstration 
designs tested in the participating states, learn about the challenges the states faced and 
the lessons they learned during the implementation process, and assess the extent to 
which full conformance with JOBS was achieved, the Department of Agriculture 
required each participating state to conduct its own evaluation of the demonstration 
implementation and process, using an independent evaluator. In addition, states were 
required to conduct evaluations of demonstration costs and impacts. The Department 
selected Social Policy Research Associates (SPR) as the national evaluation and 
technical assistance contractor to assist the states in conducting their local evaluations 


and synthesize the evaluation findings across the participating states. 


Over the last four years, SPR has reviewed state evaluation plans and provided 
suggestions to increase the soundness of individual state evaluations and promote 
consistency and comparability across states in four evaluation areas. Each state is 
completing: (1) an implementation evaluation describing the challenges faced during 
demonstration planning, start-up, and operations and how they were resolved; (2) a 
process evaluation describing key demonstration features and operational procedures 
and how they differ from the E&T program in place prior to the demonstration or in 
non-demonstration counties; (3) a cost evaluation assessing how total and per 


participant costs changed as a result of the demonstration; and (4) an impact evaluation, 


assessing how participant outcomes have changed as a result of the demonstration. 
This report comprises the synthesis of findings from the state implementation and 
process evaluations. A separate report scheduled for completion in the spring of 1997 
will synthesize the findings from the state cost and impact evaluations. 


The key questions addressed by the implementation evaluations included the 


following: 


e Is it feasible to achieve conformance between the Food Stamp E&T 
program and the JOBS program? 


e What changes were necessary to implement the demonstration in each 
state? 


e What barriers or problems were encountered and how were they 
resolved? 


The key questions addressed by the process evaluations included the following: 


e How did the demonstrations affect E&T participation patterns? 
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— What were the client targeting and selection policies and 
practices under the demonstrations and how did they differ from 
policies and practices in non-demonstration or pre-demonstration 
sites/periods? 


— What were the characteristics of demonstration participants and 
how did they differ from the characteristics of E&T participants 
in non-demonstration or pre-demonstration sites/periods? 


— How did clients flow through demonstration services and how 
did client flow differ between the demonstration and non- 
demonstration or pre-demonstration sites/periods? 


e How did the demonstrations affect the utilization of E&T services? 


— What were the services available to demonstration participants 
and how did they differ from the services available in non- 
demonstration or pre-demonstration sites/periods? 


— What were the patterns of service utilization under the 
demonstration and how did they differ from the patterns in non- 
demonstration or pre-demonstration sites/periods? 


— To what extent did demonstration participants utilize educational 
and vocational training services through non-financial referrals 
and how did this differ from non-demonstration or pre- 
demonstration sites/periods? 


OVERVIEW OF THE REPORT 

The contents of this report are based on written evaluation reports prepared by 
the demonstration states and their independent local evaluators, as well as on 
information collected by SPR over the course of our four-year technical assistance 
efforts on behalf of the demonstration states. During the demonstration period, SPR 
staff visited state administrators and observed local site operations to familiarize 
ourselves with demonstration designs and operations. However, the states and their 
independent evaluators provided the definitive detailed information on the individual 
state demonstrations. We acknowledge their hard work in conducting the state 
evaluations from which the findings in this report are drawn. 


In Chapter II, we review the findings from the state implementation evaluations. 
This chapter describes how the demonstration states prepared for demonstration start- 


9 A list of references at the end of this report details the state evaluation reports on which this 
synthesis report is based. 
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up, the challenges they encountered during the planning and operational phases of the 


demonstration, and how the challenges were resolved. 


In Chapter III, we review the findings from the state process evaluations. This 
chapter describes the key features of the demonstrations in terms of participation 
policies and patterns, available services, and service utilization patterns. For each of 
these key project dimensions, we describe the demonstration features as well as the 
demonstration context, in the form of the E&T programs that were in place prior to the 
demonstrations and the features of E&T operations in non-demonstration counties. 


In the final chapter, we discuss the implications of the process and 
implementation evaluation findings, including the feasibility of conforming E&T and 
JOBS, the strengths and weaknesses of the alternative approaches to conformance taken 
in the different states, and how the lessons learned from the demonstration might apply 
in the rapidly changing context of welfare reform and the consolidation of workforce 
development services across multiple programs and funding streams. We also discuss 
how the findings from the process study will influence and support the interpretation of 
findings from the state cost and impact evaluations. 


As an appendix to this report, we have prepared summaries of the key features of 
the conformance demonstrations in each of the participating states. We invite the 
reader to consult these state profiles for expanded discussions of state-specific issues 


summarized in the rest of the report. 
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Il THE IMPLEMENTATION EXPERIENCE 


INTRODUCTION 


As part of their evaluations of the conformance demonstration, each state 
conducted an implementation analysis designed to: (1) document the changes made to 
conform the operation of the Food Stamp E&T program to JOBS rules and regulations, 
and (2) describe any barriers to implementation and how they were addressed. Early 
results from each state’s implementation evaluation were available to assist the states in 
fine-tuning their implementation designs. The implementation findings synthesized in 
this report are intended to assist FCS and all states in deciding whether E&T/JOBS 
conformance is feasible and how to overcome federal and state barriers to consolidation 
of welfare-to-work programs. 


Methods used by the states to conduct their implementation evaluations varied. 
In Georgia, South Dakota, and Missouri, staff from the state agencies responsible for 
administering the demonstration designed and conducted in-house implementation 
evaluations. In Hawaii and Texas, outside evaluators were hired to design and conduct 
implementation evaluations.! Across the demonstration states, the data collection 
methods at the state level included reviews of written program rules and regulations 
and discussions with key staff responsible for different aspects of the demonstration. 
Information about the local implementation experience was collected through telephone 
or face-to-face discussions with staff in selected service sites, or, in two states, by the 
completion of written questionnaires by local site staff. Implementation evaluators also 
reviewed statistical reports summarizing participation patterns and service utilization 


over time. 


PLANNING, PREPARATION, AND START-UP 

The demonstration states varied widely in the number of counties and proportion 
of their statewide E&T programs included in the conformance demonstration. 
Administrators in three of the five states (South Dakota, Hawaii, and Missouri) viewed 
the demonstration, at least initially, as an initiative to transform their entire state E&T 


1The outside evaluator in Hawaii provided the state with an interim process evaluation report, but 
was not able to complete the evaluation because she moved from the state. The process report in Hawaii 
was completed by staff within the Department of Human Services. 
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system.2 South Dakota included all 20 counties with operating E&T services in the 
demonstration from the beginning. Hawaii initiated the demonstration in its largest 
county (Oahu), which had about 70 percent of the state’s Food Stamp cases, and 
planned to phase in the rest of the state during subsequent years. Missouri also 
planned to expand the demonstration over time as part of a plan to consolidate E&T 
and JOBS operations statewide. However, because of state budget cuts (in Hawaii) and 
changes in the demonstration staffing arrangements combined with implementation 
delays (in Missouri), only South Dakota actually realized full statewide operation of its 
conformance demonstration. In Missouri, the 8 counties ultimately included in the JET 
demonstration accounted for about 16 percent of the state’s work registrant pool. 


Several of the states that attempted to implement the conformance demonstration 
on a relatively large geographic scale made some exceptions to full conformance 
between E&T and JOBS in recognition of the fact that the federal funding available to 
the E&T system was so much lower than the federal funding for the JOBS program. 
Both South Dakota and Missouri depended on state child-care block grants for the 
delivery of child care assistance to demonstration participants, rather than using 
demonstration funds to expand the availability of these services. In South Dakota, 
costs were also contained by exempting individuals in self-initiated training from 
mandatory participation in the demonstration and restricting access to supportive 
services to mandatory participants. Cost containment in Missouri was also achieved by 
enrolling only a limited number of demonstration participants. 


In Hawaii, the state was prepared to increase substantially the state’s financial 
investment in E&T in order to achieve enriched services through the use of a multi- 
disciplinary case management and employment services team. However, severe state 
budget woes led to the premature dismantling of the multi-agency collaborative team 
offering demonstration services in July of 1995. In retrospect, Hawaii staff said they 
might have been too ambitious when they decided to implement the demonstration in 
two service units on the island of Oahu rather than in a single unit. 


2Georgia was also interested in adopting many of the JOBS design features for its E&T program 
on a statewide basis. However, given the high cost of providing enhanced child care and other 
supportive services to demonstration participants, Georgia proposed to include only four counties in the 
official conformance demonstration. 


3Toward the end of the demonstration period, one of the counties participating in the Missouri 
demonstration requested and received permission to fund child care expenses using demonstration funds. 


In contrast, Georgia and Texas entered the demonstration as a way to test the 
feasibility of conforming E&T and JOBS operations on a much more limited 
geographic scale, in recognition of the fact that the JOBS service model would be more 
expensive to operate. Texas selected a single county representing less than 2 percent of 
the work registrants in counties with operating E&T programs. Georgia selected four 
counties representing less than one-fifth of the statewide Food Stamp caseload 
population. Within these more limited project areas, both Georgia and Texas set about 
to test full conformance between E&T and JOBS, including not only conformed service 
models but integrated staffing arrangements. Each of these states offered 
demonstration participants access to the full JOBS menu of transportation and other 
supportive services, including child care (with the exception of transitional child care 
services). 


The scope of the demonstration projects influenced not only the design and cost 
of the conformance demonstrations in the different states, but also the logistical 
challenges the states faced in preparing and starting up demonstration operations. Each 
of the demonstration states prepared for the conformance demonstration by: 

(1) redesigning program policies and procedures, (2) redesigning MIS and reporting 
systems, (3) arranging for staff to deliver demonstration services and training staff for 
their new responsibilities, and, in several cases, (4) developing new physical facilities 
or communication infrastructure for the delivery of demonstration services. 


Overall, several of the demonstration states—including Texas, South Dakota, and 
Georgia—accomplished planning for conformance between E&T and JOBS and initial 
start-up of demonstration operations with few major problems. In contrast, Hawaii and 
Missouri experienced unexpected delays in hiring or contracting for new service staff 
to provide demonstration services. Below, we describe variations in how the 
demonstration states planned for implementation, problems that were encountered, and 


how they were resolved. 


Redesigning Program Policies and Procedures 

In each of the demonstration states, JOBS policies, procedures, and forms 
provided the framework for the development of policies and procedures for the 
conformance demonstration. In Missouri, South Dakota, Texas, and Georgia, the state 
procedures manual for JOBS was adapted to perform double duty for both JOBS and 
the E&T conformance demonstration. In Hawaii, a separate PRIDE procedures manual 
was developed, patterned after the JOBS program manual. 
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In South Dakota and Missouri, which maintained separate E&T and JOBS 
operations under the demonstration, the state JOBS procedures manuals were revised to 
cover any residual differences between the E&T and JOBS programs. For example, in 
Missouri, the procedures manual indicated that E&T demonstration participants were 
not eligible for demonstration-funded child care assistance, but should be referred to 
state block-grant child care providers. In South Dakota, planning staff prepared 
separate instruction guides and “desk aids” for use by the field staff responsible for the 
E&T demonstration and JOBS. 


In Texas, an interagency policy review team developed procedures for serving 
E&T and JOBS participants as part of an integrated BOND caseload receiving 
consolidated services in a single demonstration county. Case managers and contractor 
staff providing employment and training services to BOND participants generally did 
not have to pay attention to which assistance program (AFDC or Food Stamps/non- 
public assistance) had referred individuals to the BOND program, because E&T and 
JOBS participants received the same BOND services following JOBS policies and 


regulations. 


The demonstration states required different amounts of lead time for planning the 
demonstration. The conformance demonstration was operational in two states (South 
Dakota and Georgia) by April 1, 1993, only six months after the demonstration awards 
were announced by the U.S. Department of Agriculture. In contrast, staff in Texas 
carried out an extended planning period that called for active participation by state and 
local interagency teams in demonstration planning. The longer lead time in Texas also 
permitted the state’s independent evaluator to capture “baseline” measurements of the 
pre-demonstration system. Demonstration start-up in Texas occurred as planned on 
October 1, 1993, twelve months after the demonstration announcement. 


Texas staff, in particular, identified an extended planning period with strong 
interagency participation as a factor that supported the subsequent smooth 
implementation of the BOND demonstration in Texas. In Texas, the Texas 
Employment Commission (the primary E&T contractor) was a full partner in the 
planning process along with the Texas Department of Human Services. Both Texas 
and Georgia staff emphasized the importance of involving both state and local 
demonstration staff in the planning process to secure staff “buy in” to the overall 
concept of the demonstration at all levels and to brainstorm and address potential 
implementation difficulties before they arose. 


II-4 





Although Hawaii also convened an interagency task force prior to implementing 
the conformance demonstration, this state experienced some start-up difficulties in 
forging a unified interagency team for the delivery of E&T services at the local level. 
Local field staff provided under a contract with the Hawaii Department of Labor and 
Industrial Relations (DLIR) had traditionally been the sole service providers of E&T 
services prior to the demonstration. At the outset of the demonstration, these staff had 
some difficulty adjusting to the fact that they were now part of an interagency team in 
which DLIR and Department of Human Services staff shared service delivery 
responsibilities as part of an integrated PRIDE services unit. 


Missouri experienced substantial start-up delays after the state legislature failed to 
approve the creation of new case management positions within the Department of 
Social Services to serve demonstration participants. As described below under staffing 
arrangements, Missouri ultimately developed contracts with two different outside 
agencies to provide staff for the demonstration. Once they were hired, these contract 
staff utilized revised JOBS procedures manuals to guide their day-to-day operations. 


Redesigning MIS and Reporting Systems 
MIS for Demonstration Participants 

In addition to planning for the implementation of demonstration policies and 
procedures, demonstration planning teams had to develop management information 
systems (MIS) and reporting procedures for their conformance demonstrations. Four 
of the five demonstration states (Missouri, Hawaii, Texas, and South Dakota) adapted 
their statewide JOBS MIS and reporting systems for collecting and reporting 
information on conformance demonstration participants. South Dakota and Hawaii also 
continued to rely on MIS systems maintained by their E&T contractors (their respective 
state departments of labor) to capture information on services and outcomes for 
demonstration participants. In contrast, Georgia developed a new PC-based MIS 
system that included information on all work registrants screened for participation in its 
PEACH JET demonstration. 


MIS development and implementation for the conformance demonstration was a 
relatively smooth process in Texas and South Dakota, because the MIS systems 
developed for JOBS were already up and running. Once a few revisions were made 
(e.g., the addition of a new code to distinguish E&T from JOBS participants), these 
systems could support the data collection and reporting functions for the demonstration. 
However, in both Hawaii and Missouri, the state’s JOBS information systems were 
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undergoing major development or revision at the time the demonstration was being 
planned. As a result, in both states, the conformance demonstration was implemented 
before the automated MIS system was operational. This created an extra paperwork 
burden for demonstration service staff, who had to maintain “hard copy” records until 
the automated system was available. It also hampered policy and administrative staff, 
who lacked access to summary reports about demonstration operations during the 


critical start-up phase. 


Because administrative staff in Georgia developed a new MIS system for the 
demonstration, they were not dependent on outside systems to accommodate the 
information collection and reporting needs of the demonstration. However, the 
development of a new system created its own challenges, including the training of staff 
to use the new system and the development of quality review procedures to ensure that 
the system provided meaningful information. The development of a totally new 
information system for the demonstration also hampered the efforts of the state’s 
impact evaluator to obtain comparable data for participants in demonstration and non- 


demonstration counties. 


MIS for Referral of Targeted Work Registrants 

In addition to developing an information system to report on demonstration 
participants once they were enrolled, each state had to develop an automated system to 
support the recruitment and referral of targeted work registrants into demonstration 
services. During the planning phase, each demonstration state developed a procedure 
to identify target group members and inform service providers about potential 
participants, their priority group status, and their status as voluntary or mandatory 
work registrants. The resulting systems accomplished their overall objectives, but 


several operational problems remained. 


Minor problems with automated referral procedures were easily resolved. Initial 
referral lists provided to E&T staff in South Dakota were not sorted by target group 
and were difficult for case management staff to use in deciding whom to call in for 
services; subsequent lists were sorted by target group. In Georgia, there was some 
initial confusion among eligibility workers about assigning target group status. This 
problem was resolved through additional staff training. 


Other problems with the operation of automated referral systems could not be so 
easily resolved. One problem, experienced in a number of demonstration states, was 
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the high rate of inappropriate referrals. Unlike the referral of targeted JOBS 
participants, whose benefit status was likely to remain stable over a number of months, 
referrals of targeted E&T work registrants included high proportions of individuals 
who were no longer eligible for services by the time they were called in. Because of 
the low response rate to call-ins and the high rate of exemptions among individuals who 
did show up in response to call-in notices, staff in Hawaii’s PRIDE program began to 
check lists of referrals manually against the state’s Food Stamp benefits automated 
system before sending out call-in notices. In Georgia, where referrals were not 
screened by hand before call-in, over 50 percent of those who responded to call-in 
notices were found to be ineligible because they were no longer receiving Food Stamps 
or were exempt. The implementation of a selective targeting policy for E&T was 
based on the assumption that automated records could be used to identify participants 
who would be most able to benefit from demonstration services. The high rate of 
inappropriate referrals experienced in the demonstration states calls that assumption 
into question. 


Identifying Staffing Arrangements and Training Demonstration 
Staff 


Developing arrangements with contractors for the staffing of demonstration 
services and/or hiring in-house staff were two of the biggest implementation tasks 
facing the demonstration states. Once demonstration staff were identified, staff training 
was a key to demonstration success. Staff turnover at the state and local level provided 
an unexpected challenge in several states, particularly in states where the positions 
were categorized as temporary jobs, due to the limited period of demonstration 
operations. 


Staffing Arrangements 
In South Dakota, Georgia, and Texas, demonstration services were staffed using 
already existing delivery systems and/or contractual relationships. South Dakota 
retained the state Department of Labor (its previous E&T contractor as well as the state 
JOBS contractor) as its contractor for the demonstration. In South Dakota, local Job 
Service offices continued to designate one or more specific staff members to serve E&T 
participants. 


4Prior to the implementation of manual screening, during one reporting period 1,197 out of 1,339 
referrals were determined to be exempt or had become ineligible for Food Stamps. 
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In Georgia and Texas, the service delivery structure built on existing JOBS 
service delivery systems, into which E&T operations were integrated. In Georgia, 
local welfare offices were instructed to consolidate JOBS and E&T operations starting 
in FY 1993. This consolidated delivery system was used for the delivery of JOBS and 
E&T services statewide, and provided the framework for staffing the demonstration in 
the four demonstration counties. In Texas, the demonstration was used to expand the 
existing JOBS delivery system, which depended upon the coordinated delivery of 
services by case managers from the Department of Human Resources (DHR) and 
contracted employment facilitators from the Texas Employment Commission (TEC).° 
DHR created additional case manager positions to meet the needs of the combined 
JOBS and E&T caseloads under the BOND demonstration. To promote interagency 
coordination and the delivery of seamless services to BOND participants, DHS group 
case managers were co-housed at the same facility as TEC employment facilitators. In 
both Georgia and Texas, a number of the staff who had previously served E&T 
participants were absorbed into the consolidated JOBS/E&T systems. 


In Missouri and Hawaii, entirely new service delivery arrangements were 
developed for the operation of the conformance demonstration. In Missouri, after the 
state legislature declined to fund new case management positions within the Department 
of Social Services (DSS), DSS turned to administrators of two JTPA service delivery 
areas (SDAs) as outside contractors for the provision of case management staff to 
operate the demonstration. During an initial program year, the state Department of 
Jobs and Training Development was an intermediary in the contracting arrangement. 
Subsequently, DSS contracted directly with the local SDAs. A total of three case 
management positions were created for provision of JET services in the eight 
demonstration counties. New staff were recruited and hired into these positions. 
Although staff were nominally employees of the local SDAs, primary staff supervision 
and training was provided by supervisors in the local DSS JOBS unit. The DSS 
demonstration administrator explained that although they are not officially DSS staff, 
“the JET case managers think, act, and work like JOBS case managers.” 


In Hawaii, the Department of Human Services hired new staff or reassigned 
interested existing staff to newly created staff positions in two local demonstration 


5Now the Texas Workforce Commission, under a significant statewide governmental 
reorganization plan. 
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service units. Staff positions in the PRIDE units included two unit supervisors, ten 
Case managers, two social workers, and two secretaries. These new staff were part of 
a larger interdisciplinary team also consisting of employment counselors assigned to the 
demonstration under a contract with the state Department of Labor and Industrial 
Relations and public health nurses available on a consultant basis. However, because 
positions in the demonstration unit were identified as time-limited positions, it was 
more difficult than usual to fill these jobs. 


Staff Training 


All demonstration states prepared staff for their new jobs by providing them with 
training developed for JOBS personnel, including training in JOBS policies and 
procedures and forms completion. In most states, this included attendance at an initial 
1 to 3 day training session, followed by periodic local office training sessions as well 
as attendance at annual training sessions. In states with consolidated E&T and JOBS 
operations (Texas and Georgia), experienced JOBS staff also provided peer and 
supervisorial support for new staff inexperienced with JOBS. In Missouri, even though 
the JET case managers were not employed by DSS, the state informally assigned them 
to the local JOBS unit, where they received supervision and consultation from the 
JOBS unit supervisor and participated in staff meetings and training conferences. This 
arrangement provided a professional support network for JET staff. 


In their implementation evaluation reports, several states emphasized the 
importance of including a wide variety of agencies and staff from a variety of positions 
in the staff training sessions held prior to demonstration start-up. For example, Texas 
included in BOND training for all Department of Human Services (DHS) income 
maintenance staff, as well as DHS case management staff and Texas Employment 
Commission employment facilitators, so that all staff could understand how each of 
their roles affected the entire service system. Georgia also emphasized the importance 
of involving eligibility workers in training to ensure that they would be committed to 
the program, knowledgeable about the services available through the demonstration, 
and able to present a positive first impression to mandatory registrants and potential 
volunteers. Eligibility workers in Georgia were also described as key players in 
providing E&T operators with updated information about household benefits and 
earnings status. In addition, their cooperation was needed for effective follow-through 
on requests for sanctions. 


Texas and Hawaii, both of which developed multi-agency teams for the delivery 
of demonstration services, also emphasized the importance of cross-training staff in 
multiple roles to develop a “team approach” among the different agency participants. 
For example, the employment counselors in Hawaii’s PRIDE program experienced 
some difficulties working as members of the multi-agency multi-disciplinary team 
developed for the demonstration, since they had previously been used to being the sole 
E&T staff responsible for all aspects of program services. In Texas’s BOND program, 
Department of Human Services case managers and Texas Employment Commission 
employment facilitators were both perceived by clients as playing important customer 
support functions. Texas staff identified the largest single challenge faced by the 
integrated services team as “converting initially resistant clients into willing 
participants through open communication and articulation of the potential benefits of 
the program in which staff believed.” Meeting this challenge required a high level of 
staff morale as well as good relations among staff and between staff and clients. 


The importance of good communication and “people skills” was referenced by a 
number of the programs as a prerequisite for effective staff functioning under the 
demonstration. Rather than processing large numbers of participants through a 
standardized service sequence, as had occurred under the pre-demonstration E&T 
design in most states, demonstration staff were called on to establish personal 
relationships with participants and support them in developing individual career plans 
and goals. These counseling and coaching skills did not come naturally to all staff who 
had been carried over from the previous E&T program, but had to be supported 
through staff training. 


Problems with Staff Turnover 

Staff turnover after the start-up of the demonstration created a significant problem 
in several states. Hawaii’s PRIDE units were particularly susceptible to staff turnover 
because the program was staffed by new positions that had been designated by the state 
as “limited term appointments.” Over the course of the demonstration, the program 
lost a number of case managers, one social worker, and at least one secretary due to 
staff turnover. Staff turnover was also referenced as a problem in the implementation 
evaluation reports prepared by Georgia and Missouri. 


Staff turnover created several challenges for the conformance demonstrations. 
First, turnover created a need to repeat staff training starting with the basics of 
program design and procedures. Second, it made it difficult for some of the 
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demonstrations to service and replenish their caseloads and limited the number of 
participants ultimately served. In Hawaii, the loss of a social worker created a 
“bottleneck” in services at a time when social worker appointments were already 
backlogged with a two-week delay. The loss of a unit secretary at another point made 
it difficult for staff to send out call-in letters and other required communications. 


Developing Facilities and Equipment to Support the Delivery of 
Services 


Several states found that the development of new facilities was essential to the 
implementation of the demonstration. The implementation of the demonstration in 
Hawaii required the two combined Department of Human Services/Department of 
Labor and Industrial Relations PRIDE units to relocate to new facilities because of the 
number of staff that had to be accommodated in each unit. As part of preparation for 
demonstration start-up, administrative staff were involved in securing office space, 
leasing furniture, purchasing computer equipment, and arranging for the installation of 
telephone lines. 


In Texas, the two agencies involved in the implementation of the BOND 
demonstration—the Department of Human Resources and the Texas Employment 
Commission—also leased new space to house the BOND staff assigned from both 
agencies. The selection of a service site located in an attractive commercial office 
setting was perceived to be important in communicating the positive “real world” work 
orientation of the BOND demonstration program to participants. 


In Missouri’s “Bootheel” area, two case managers were responsible for a seven- 
county service area. The state arranged for these staff to work out of their homes and 
provided them with a communication infrastructure to support their “roving” mode of 
operations. To enable them to communicate with Department of Social Services (DSS) 
and local JTPA Service Delivery Area (SDA) staff at a distance, JET case managers 
were given access to electronic mail at the DSS field offices. To enable them to 
communicate with clients and other agencies while on the road, they were provided 
with cellular phones. In addition, an “800” telephone number was established to 
enable clients to get in touch with the case managers without giving out their home 
phone numbers. 
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PROGRAM OPERATIONS 

All five demonstration states appeared to be successful in operating programs that 
realized their intended service designs. The conformance of E&T and JOBS policies 
and procedures under the state demonstrations greatly simplified the jobs of local staff 
who had to work with the participants of both programs. However, the demonstrations 
inadvertently caused increased confusion for E&T staff who had the misfortune to be 
responsible for serving E&T clients in both demonstration and non-demonstration 
counties, as was the case in some local offices in Missouri and Georgia. 


Demonstration operations were perceived by most staff as providing more 
meaningful services to participants who were more motivated than under the previous 
E&T program. Texas, which tested the feasibility of consolidating E&T and JOBS 
service delivery, reported that there appeared to be few disadvantages and multiple 
benefits to operating workshops serving both groups. Staff reported that positive group 
interactions occurred among the diverse participants in its job readiness workshops. 
Operational challenges faced by some states included: (1) difficulties in recruiting the 
desired number or type of participants; (2) difficulties serving the recruited 
participants; and (3) problems matching participants to appropriate services. The 
demonstration also appeared to fail to accomplish its goal of reducing nonparticipation 
among E&T work registrants. 


Difficulties in Recruiting the Desired Number of Participants 


Each of the demonstration states reported that it continued to experience low 
response rates to call-in notices sent to mandatory work registrants; some reported only 
limited success in efforts to recruit volunteers. Texas staff, as reported in the state’s 
process and implementation evaluation report, called for “more and better marketing of 
program activities and support services” to correct this problem. In particular, Texas 
reported difficulties recruiting less job-ready clients for participation in basic education 
and preemployment training components. The problem of large numbers of 
inappropriate referrals, discussed above, also made it difficult for demonstration staff 
to fill available service slots in Hawaii. To fill these slots, most of the demonstration 
states had to conduct outreach to a much larger number of work registrants than they 
ultimately had the capacity to serve. 


Difficulties in Serving the Targeted Participants 


Despite the large number of “no-shows,” the more intensive and individualized 
intake and assessment activities and services provided as part of the conformance 
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demonstrations made it difficult for states to call in all mandatory work registrants, or 
even all targeted work registrants. This situation was not perceived as problematic in 
states such as Missouri or Hawaii, each of which wanted to enroll a limited number of 
hard-to-serve work registrants into more intensive services. However, South Dakota, 
Georgia, and Texas were eager to reach a significant proportion of the targeted work 
registrant population, particularly after an initial start-up period. In Georgia, all four 
demonstration counties found they had the capacity to call in all new mandatory target 
group referrals each month; one demonstration county called in all referred mandatory 
work registrants. South Dakota was particularly interested in continuing to call in all 
mandatory work registrants. When its largest urban county found that it did not have 
enough E&T staff to call in all work registrants, the state arranged for the assignment 
of additional demonstration staff to that county. 


Difficulties in Matching Participants to Appropriate Services 


In Texas, although services developed for the JOBS clientele were generally well 
received by E&T participants in the BOND demonstration, two problems were 
experienced in matching demonstration participants to appropriate services. The first 
problem occurred with “life skills” training, which had been developed for a largely 
female JOBS caseload. Although efforts were made to adapt this curriculum for both 
genders under the demonstration, males did not find this curriculum very relevant to 
their life concerns. A second difficulty arose as a result of a scarcity of local 
vocational skills training resources appropriate for “non-college-bound” individuals. 
Case managers did not have much to offer JOBS or E&T participants who lacked 
specific vocational skills but for whom training programs in an academic setting (e.g., 
a two-year certificate program at a local community college or vocational technical 
school) was not realistic. 


Georgia reported that work registrants under age 24 who lacked a high school 
diploma were particularly difficult to match to educational services, as required by 
JOBS rules. Operational problems resulting from this requirement were two-fold. 
First, not all demonstration counties had available local resources to support attendance 
by control group members in educational programs.° Second, participation in 


6In the four demonstration counties in Georgia, expanded supportive services (including tuition 
payments) were available to treatment group members only. Control group members might be required 
to attend education services under the demonstration but were ineligible for tuition support payments. 
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educational activities frequently conflicted with participants’ desire for employment 
rather than education. As a result, mandatory participants were sometimes disruptive 


in class. 


Georgia administrative staff also expressed some surprise at the large number of 
demonstration volunteers who appeared to be motivated primarily or solely by the 
desire to access child care services while in training or during the first ninety days of 
employment. The size of this group and the significant proportion of demonstration 
resources they consumed raised questions about the allocation of resources across 
different target groups. 


Some demonstration staff in Georgia were also concerned to achieve equity of 
services between treatment and control group members. The state reported that, rather 
than denying education and training opportunities to control group members, case 
managers in some counties reserved all “free” educational resources for control group 
members and used the expanded supportive services funding available under the 
demonstration to pay tuition for education or training for treatment group members. 
While this did not result in the provision of inappropriate services to demonstration 
participants, it may have undermined the distinction between treatment and control 
group services that the state was trying to maintain for evaluation purposes. 


Disappointing Results of JOBS Sanctioning Procedures 


Implementation of strengthened sanctioning provisions under the demonstration 
does not appear to have resulted in improved program participation rates among E&T 
participants. Instead, demonstration sites continued to experience high no-show rates in 
response to call-in notices. Staff in several states hypothesized that the strengthened 
sanctioning provisions, in combination with required participation in longer or more 
intensive services, may have caused some Food Stamp work registrants to choose to 
terminate their Food Stamp benefits rather than participate in demonstration services. 


The conformance of JOBS and E&T sanctioning procedures also had an 
unanticipated effect on JOBS participants. All demonstration states reported that the 
conformance of Food Stamps and JOBS work requirements and sanctioning policies 
had enabled them to include both Food Stamp benefits and AFDC benefits in the 
sanctions for noncompliance among JOBS participants. In contrast to the lack of the 
hoped-for effect on E&T participation rates, this substantial toughening of the 
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sanctioning provisions for JOBS participants was perceived to have been a significant 
factor in improving participation in the JOBS program. 


SUMMARY 


The implementation experiences of the five demonstration states indicate that 
conformance of the rules and regulations of the E&T and JOBS programs is feasible. 
States that tested conformance on a large geographic scale sometimes had to place 
limits on the services available to participants (particularly supportive services) to 
constrain demonstration costs. States that tested conformance on a more limited basis 
were able to achieve full conformance and consolidation of JOBS and E&T operations. 
Demonstration operations were perceived by staff in most states as having improved 
the ability of the E&T program to provide meaningful services to motivated 
participants. 


The demonstrations were able to adapt policies and procedures as well as MIS 
and reporting systems developed for the JOBS program. These systems worked well 
when they were already operational. In several instances, however, the demonstrations 
were inconvenienced because JOBS information and reporting systems were in the 
process of development or revision. Staffing arrangements were particularly time 
consuming in the two states that created new service delivery systems for the 
demonstration. In several states, staff turnover after demonstration start-up created 
operational challenges. 


Perhaps the most striking operational issue identified by the implementation 
evaluation was the widespread difficulty in identifying and recruiting appropriate 
participants. In all participating states, a low percentage of mandatory work registrants 
responded to call-in notices and a substantial percentage of the referred work 
registrants were found to be already off Food Stamps or exempt by the time their status 
was assessed by work program staff. Demonstration staff hypothesized that the low 
response to call-in notices by mandatory work registrants under the demonstrations 
reflected both the rapid turnover of the Food Stamp caseload in the normal course of 
events and the deterrent effect of longer or more intensive job search requirements and 
stronger sanctions for nonparticipation in E&T activities.’ 


7The impact evaluations will not be able to shed much light on the deterrence hypothesis because, 
in all but one demonstration state, they were limited to assessing how the demonstration affected 
outcomes for individuals who actually received demonstration services. 
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In the next chapter, we describe how the demonstration transformed the 
participation patterns and services provided to participants. The cost and impact 
evaluations currently underway in each of the demonstration states will address whether 
conformance resulted in improved outcomes for participants or reduced costs to 
taxpayers. 
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Iii FEATURES OF THE CONFORMANCE 
DEMONSTRATIONS 


INTRODUCTION 


The process evaluations conducted by each demonstration state described the 
demonstration design and operations within the demonstration site(s) as well as the 
E&T system that would have been in place in the absence of the demonstration. Texas, 
Georgia, and Hawaii identified specific non-demonstration counties as “comparison 
counties” for the purpose of describing the non-demonstration E&T context.! In 
Missouri, the balance of the state was selected for comparison with the eight 
demonstration counties. Because South Dakota had included its entire statewide E&T 
operations in the demonstration, the pre-demonstration period was used to provide a 
point of reference for the demonstration. 


In Georgia, South Dakota, and Missouri, state agency staff completed the process 
evaluation reports. Process reports in South Dakota and Georgia were based on 
information about local demonstration operations obtained from written documents and 
responses to questionnaires completed by direct service staff and their supervisors, 
supplemented by on-site observations by state office administrative staff. In Missouri, 
information for the process evaluation was compiled by a new state administrator based 
on written materials and telephone discussions with demonstration staff and key state- 
level actors. In Hawaii and Texas, outside evaluators conducted extensive field 
observations and discussions with state and local staff and prepared detailed 
descriptions of program design and operations. 


In synthesizing the information from the process evaluations, SPR has tried to 
highlight both common features and differences across the demonstration states. The 
five states participating in the Food Stamp E&T/JOBS conformance demonstration 
shared the goals of simplifying administrative procedures across different welfare-to- 
work programs, focusing limited E&T program resources on individuals most likely to 
benefit from services, and expanding the range and intensity of E&T services available 
to Food Stamp work registrants. All five states used the JOBS program design as the 


1 The comparison counties also provide a framework for comparing program costs and outcomes 
between demonstration and non-demonstration operations in the cost and impact evaluations. 
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model for redesigning E&T work requirements, client targeting and selection 
procedures, and service offerings. There were significant differences, however, in the 
details of the conformance demonstrations implemented in the five states. In this 
chapter, we review the key features of the state demonstrations in the areas of (1) client 
targeting policies and resulting participation patterns and (2) service design and 
utilization. For each program element, we discuss how the demonstration designs 
differed from the E&T programs in place prior to the demonstration as well as from the 
non-demonstration E&T programs operating during the study period. 


CLIENT TARGETING POLICIES 

The State Context for the Demonstrations 

Prior to FY 1992, all five participating states operated E&T programs that 
attempted to reach the universe of mandatory work registrants. These broad client 
targeting policies were driven by the U.S. Department of Agriculture’s participation 
rate performance standard, which required states to enroll or issue notices of 
noncompliance to at least 50 percent of all mandatory nonexempt work registrants.” 
Although permitted by federal law, participation by volunteers (individuals exempt 
from work registration) was not emphasized. 


Beginning in FY 1992, the Department of Agriculture substantially modified its 
participation rate performance standard, by lowering the national performance standard 
to 10 percent. This policy change recognized that universal targeting had resulted in 
the delivery of low-cost/low intensity E&T services. The modified performance 
expectation was intended to give individual states the latitude to implement more 
intensive services by targeting services more narrowly to specific subgroups within the 
work registrant population. As a result of the change in federal policy, a number of 
states amended their statewide E&T plans to reach a smaller subset of the total work 
registrant population. Among the five demonstration states, formal statewide targeting 
policies were amended in Georgia and Texas. 


e Beginning in March 1993, Georgia implemented selective targeting for 
the E&T program on a statewide basis using the priority target groups 
developed for the PEACH JET demonstration (described below). 


2 The actual formula for computing participation rates under the E&T program gives equal credit 
for entry into an E&T service component and issuance of a “notice of adverse action” for failure to 
respond to a mandatory call-in notice or failure to comply with the participation requirements of an 
assigned component. States meet the performance standard through a mix of services and sanctions. 
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According to the new targeting policy, mandatory work registrants who 
belonged to six priority target groups were called in for services before 
members of non-target groups. Members of non-target groups could be 
called in on a space-available basis. 


e Starting in FY 1993, Texas implemented a statewide selection procedure 
for E&T (following a system already implemented for JOBS) that sorted 
work registrants into “service levels” based on their previous education 
and work histories. Work registrants with at least eighth grade 
completion and some recent work experience were referred to E&T 
services, while work registrants with significant employment barriers or 
less than an 8th grade education and no recent work history or training 
were not required to participate. 


Although it did not specify any particular target groups for its statewide E&T 
program, Missouri also amended its non-demonstration participant selection and call-in 
practices in response to the reduced federal participation rate standard. In Missouri, a 
random selection procedure based on social security numbers was implemented to 
determine which mandatory work registrants are required to participate. Thus, three of 
the five demonstration states modified their client targeting and selection procedures 
independently of the E&T/JOBS conformance demonstration in response to the 
significantly lowered federal participation rate standard for E&T. In the non- 
demonstration counties in Hawaii, all mandatory work registrants continued to be 
referred to the E&T program and called in for services. (In South Dakota, all counties 
operating E&T services were included in the demonstration.) 


Demonstration Policies 


The client targeting designs encouraged as part of the E&T/JOBS conformance 
demonstration were distinguished by two features: (1) selective targeting based on the 
designation of priority target groups and (2) priority given to participation by 
volunteers. The JOBS program emphasized services to groups whose members were 
most at risk of long-term welfare participation. For the AFDC population served by 
JOBS, these groups included individuals under age 24 without high school completion 
or equivalency, individuals under age 24 with little or no work experience, and 
individuals who had received AFDC benefits for 36 or more of the 60 months prior to 
certification. 


Target Group Policies 


In conforming the Food Stamp E&T program to the JOBS model, four of the five 
participating states identified specific high-risk target groups. Some demonstration 
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States tried to mirror the JOBS priority target groups as closely as possible, while other 
states adapted the JOBS priority categories to make them fit the special characteristics 
of the Food Stamp work registrant population (e.g., by targeting homeless individuals, 
as in Hawaii) or in order to address other management priorities (e.g., by targeting 
absent parents with child support claims, as in Missouri). As described above, Texas 
implemented a statewide procedure that excused from participation the least job-ready 
work registrants (operationalized as individuals with serious employment barriers or 
less than an eighth grade education with no recent work experience). However, the 
Texas BOND demonstration did not establish any “high risk” priority groups within the 
remaining work registrant population, although it designed a range of services 
appropriate for less job-ready as well as more job-ready participants. 


Exhibit III-1 summarizes the different priority groups targeted for E&T 
participation by the demonstration states. With the exception of Texas, all states 
included youth under 24 with limited education and/or work experience. All four states 
other than Texas also targeted long-term Food Stamp recipients, although they each 
operationalized “long-term” receipt differently. Three states targeted work registrants 
of any age with limited work experience. Beyond these common targeting patterns, 
three states identified additional priority groups. Hawaii added several target groups 
whose members have particular barriers to employment (e.g., homeless individuals and 
individuals with a primary language other than English). Georgia and Missouri 
targeted male wage earners with families to support (primary wage earners in two- 
parent households in Georgia and absent parents with child support claims in Missouri). 
Missouri targeted individuals who had recently lost eligibility for AFDC/JOBS benefits. 
Georgia reached out to “self-initiated” work registrants already enrolled in education or 
training programs. 


Some states designated all target groups as having equal priority for 
demonstration services (Hawaii, Missouri); others established relative priorities among 
the different target groups (South Dakota, Georgia). Some states indicated that they 
would call in all work registrants from target groups before serving non-target group 
members (Missouri, South Dakota, Georgia); others indicated that target group 
members should comprise at least a minimum percentage of all work registrants called 
in for services (Hawaii, where 55 percent of all work registrants called in for E&T 
services were supposed to be from target groups). As described in the next section, the 
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Exhibit ITI-1 
Target Groups Established by States 


Target Group 


Under age 24 without high school diploma 
or equivalent 




























Under age 24 with little or no work 
experience in the last twelve months 


No high school diploma or equivalent 
Little or no work experience 


At least eighth grade completion and some 
recent work experience 


Received Food Stamps 36 out of the last 
60 months 


Received Food Stamps 24 out of the last 
36 months 





Received Food Stamps 12 out of the last 
24 months 


Homeless 


Primary language other than English 





Primary wage earners in two-parent 
households 


Individuals already enrolled in education 
or vocational training programs 


Absent parents with child support claims 






JOBS participants who have lost AFDC 
eligibility due to age of children or income 
from work 





Displaced homemaker or former AFDC 
recipient 
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demonstration states’ formal client targeting policies affected participation patterns in 
different ways. 


Priority to Volunteers 


The JOBS program also called for priority to be given to volunteers, including 
individuals exempt from work registration and mandatory or exempt work registrants in 
self-initiated training. The five demonstration states differed in their emphasis on 
participation by volunteers in the conformance demonstrations. South Dakota was least 
enthusiastic about the prospect of enrolling E&T volunteers, particularly students in 
self-initiated training.? To prevent the allocation of significant program resources to 
volunteers, South Dakota exempted students in self-initiated training from mandatory 
participation in the demonstration. They also restricted the availability of supportive 
services to mandatory participants, thereby eliminating the supportive services and 
child care “magnet” that attracted significant numbers of voluntary participants and 
individuals with self-initiated training plans to the conformance demonstrations in other 
states (Texas, Georgia, Hawaii). The remaining four states indicated that they would 
give priority to volunteers. As described in the next section, participation by 
volunteers exempt from work registration requirements ended up occurring most 
frequently in Missouri, Georgia, and Texas, while participation by “mandatory 
volunteers” (mandatory work registrants who volunteered prior to being called in for 
services) was a frequent occurrence in Hawaii. 


PARTICIPATION PATTERNS 
Size and Characteristics of the Work Registrant Pool 


All five demonstration states implemented the JOBS work registration rules as 
part of the Food Stamp E&T/JOBS Conformance demonstration. The shift from E&T 
work registration rules to JOBS rules resulted in the inclusion of several new categories 
of work registrants, most notably recipients of federal unemployment benefits, 
caretakers responsible for children between the ages of three and six,4 and students 
enrolled between half-time and full-time in self-initiated education or training. In 





3 Program staff indicated that they did not want to use limited program dollars to support 
education or training that would occur even without the EXT program. 


4 In most states, the JOBS program requires mandatory work registration by caretakers 


responsible for children older than three. In South Dakota, at state option, work registration was 
required for all caretakers with children older than one year. 
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addition, under JOBS rules, custodial parents under age 20 are required to participate 
even if they have young children. Two demonstration states (Georgia and Missouri) 
continued to apply E&T individual exemption criteria to the pool of mandatory work 
registrants before referring mandatory nonexempt work registrants to the demonstration 
E&T programs. These states explained that the individuals exempted from participation 
under these criteria ultimately would have received individual deferrals under JOBS 
program rules (e.g., for lack of available child care/transportation or temporary health 
problems). 


Each demonstration state expected the E&T work registrant pool to increase as a 
result of the shift from E&T to JOBS work registration rules. Only three states (Texas, 
South Dakota, and Missouri) provided information on the number of work registrants in 
the demonstration service areas during both a baseline and demonstration period.> As 
shown in Exhibit III-2, the work registrant pool increased by approximately 30 percent 
in Texas and South Dakota under the demonstration, but actually declined by 15 percent 
in Missouri. It is likely that factors other than the conformance demonstration 
contributed to the shift observed in each state. For example, in Texas, the increase in 
the work registrant pool in the demonstration county was matched by an increase of 
nearly the same magnitude in the rest of the state (which was not subject to JOBS work 
registration rules). In Missouri, staff noted that local economic conditions had 
improved between the two periods. 


The potential increase in the size of the work registrant pool under the 
conformance demonstration was perhaps less significant in the experience of the 
demonstration states than the mandatory inclusion of the new categories of: 

(1) caretakers responsible for children under the age of 6 years, and (2) students 
attending self-initiated training between half-time and full-time. As described below, 
the individuals representing the intersection of these two groups—parents with young 
children interested in pursuing education and training at the post-secondary level— 
became some of the most enthusiastic participants in demonstration services in all 
participating states except South Dakota. On the other hand, staff in some states 
worried that the new mandatory participation requirement for caretakers with young 





5 Additional information on how the demonstrations affected participation patterns may be 
included in the states’ impact evaluation reports, scheduled for completion by September 30, 1996. 
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Exhibit III-2 
Shifts in Size of Work Registrant Pool 


Demonstration State | Pre-Demonstration | Post-Demonstration 
Missouri FFY94;* 12,272 FFY95: 10,723 -15% 

4 counties 

South Dakota FFY92: 4,696 FFY94: 6,151 +31% 

20 counties 

Texas March 1993: 2,814 March 1994: 3,716 +32% 

1 county 















* FY1994 includes 8 months of pre-demonstration operations and 4 months of demonstration 
operations in Missouri. 
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children was causing some work registrants to give up Food Stamp benefits rather than 
participate in E&T services. 


Shifts in Call-In Procedures and Participation Rates 


As described later in this chapter, the services available to demonstration 
participants were more intensive, in most cases, than the E&T services provided prior 
to the demonstration or to participants in non-demonstration counties. Because of the 
more intensive service design, program call-in and enrollment procedures under the 
conformance demonstration usually differed substantially from the “universal call-in” 
procedures common prior to the demonstration. 


Call-In Procedures 


Individual assessment and case management services were key elements of the 
demonstration service designs in all five demonstration states. Because an individual 
assessment interview with an intake worker/case manager was used as the entry point 
into demonstration services in most states,® the call-in of voluntary and mandatory work 
registrants was “caseload” driven, for the most part. That is, individuals interested in 
volunteering for demonstration services and mandatory work registrants referred to the 
E&T program by income maintenance eligibility workers were placed in a work 
registrant pool or waiting list, from which individuals were called in as needed (using 
the priority targeting and call-in procedures established in that state) to fill caseload 
vacancies as they occurred. A common procedure was to enroll all volunteers and to 
implement a random selection procedure for calling in mandatory work registrants. 
Maximum caseload sizes constrained the demonstration service capacity particularly in 
Missouri (where caseloads averaged 35 to 40 per case manager) and for participants in 
education and training services in Texas (where individual case managers initially 
handled caseloads of not more than 75 participants).? Case management capacity was 
less of a determining factor in calling in targeted work registrants in Hawaii, where the 
maximum caseload size was 112, and in scheduling work registrants for job 
readiness/job search services in Texas, where the maximum “group case management” 
caseload size was 125. 


6 An exception was Texas, where participants in the employment preparation and job search track 
received “group” case management services. 


7 Over time, case managers serving individuals enrolled in post-secondary training took on larger 
caseloads in the Texas BOND demonstration, as a result of the high demand for this service. 
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No official guidelines about caseload size were established in Georgia or South 
Dakota. The service capacity of these programs was constrained by the interview 
schedule of E&T staff who met individually with demonstration participants to develop 
individual employment plans. After an initial start-up period when program caseloads 
were being established and built up from zero, the demonstration sites in South Dakota 
and Georgia called in all mandatory targeted work registrants or even, in some cases, 
all mandatory work registrants. 


Participation Patterns 


The resulting participation patterns can be characterized by the proportion of all 
mandatory work registrants that participated in demonstration services, the number and 
percentage of volunteer participants, and the extent that the demonstrations served only 
target group members or selected participants from a broader work registrant pool. In 
their local process and implementation evaluation reports, the demonstration states 
provided preliminary information on participation patterns during sample periods after 
demonstration start-up. The information provided by the states is summarized in 
Exhibits III-3 and III-4. The impact evaluation reports to be prepared by the states are 
expected to provide more extensive information on how the conformance demonstration 
affected E&T participation patterns. 


As shown in Exhibit III-3, despite variations in client targeting and call-in goals 
and procedures, each of the demonstrations enrolled relatively small proportions of the 
mandatory work registrant pool. The demonstration that appeared to reach the 
smallest proportion of work registrants was Missouri’s JET program, which enrolled 
only 123 (1.2 percent) of 10,168 mandatory work registrants in the 8 demonstration 
counties during FFY 1995. The demonstration that appeared to reach the largest 
proportion of work registrants was South Dakota’s, which enrolled 1,027 (10 percent) 
of 9,779 individuals designated as mandatory work registrants in the 20 demonstration 





8 The participation rate shown in Exhibit III-3 compares an unduplicated count of demonstration 
participants to the reported number of mandatory work registrants in the demonstration service area. 
This is very different from the official “participation rates” reported to FCS, which include the number 
of sanctions requested as part of the participation rate and double-count individuals receiving services 
from multiple components. 
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Exhibit ITI-3 
E&T Participation Patterns in Demonstration Sites 


Demonstration State | Participation Rate Volunteers as a Target Groups as a 
Among Mandatory Percentage of All Percentage of all 
E&T Work Participants Participants 
Registrants 


Georgia Exempt volunteers: 81% + 
4 counties 28% * 
Hawaii Exempt volunteers: 71% 
1 county 4%; mandatory 

volunteers: 57% 


Missouri Exempt volunteers: 56% 
8 counties 53%; mandatory 

volunteers: 8% 
South Dakota Very few About 50% 
20 counties 
Texas 5% 30 to 40%8 N/A: no priority 
1 county target groups 





*Based on information for the period 4/1/93 through 3/31/96. 
TBased on information for the period 4/1/93 through 3/31/96. 
+Based on client-level collected by local impact evaluator for period 7/93 through 6/94. 


§ The Texas evaluation contractor indicated that these include both exempt volunteers and 
mandatory work registratns who volunteered for services prior to being called in. 
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Exhibit II-4 
Shifts in E&T Participation Rates 


Demonstration State | Participation Rate Participation Rate | Difference Between 
Among Mandatory | Among Mandatory | Demonstration and 
Work Registrants in | Work Registrants in | Non-Demonstration 
Demonstration Sites | Non-Demonstration | in Percentage Points 
Sites or Period 


Georgia 
4 counties 


Hawaii 


1 county 


Missouri 
8 counties 


South Dakota 
20 counties 


Texas 
1 county 





*Based on client-level data collected by local impact evaluator for the period 7/1/93 through 
6/30/94. 


tBased on client-level data collected by local impact evaluator for the period 7/1/90 through 
6/30/91. 
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counties between July 1993 and June 1994.9 In contrast to the demonstration sites, 
which appeared to be quite selective in enrolling mandatory work registrants, the nation 
as a whole enrolled 31 percent of all mandatory work registrants in service components 
in FY 1994 according to reports provided to the Department of Agriculture by the 
states. (This statistic may be somewhat high, due to the fact that individuals placed in 
more than one component are double-counted in the Department of Agriculture 
reports.) 


Exhibit III-4 compares the proportion of mandatory work registrants enrolled in 
demonstration services to the proportion enrolled in E&T services in non-demonstration 
counties or during the pre-demonstration period. As shown in the exhibit, a smaller 
proportion of the work registrant pool was enrolled during the demonstrations, perhaps 
as a result of the lower caseload sizes and more individualized attention received by 
participants under the demonstration designs. However, even in non-demonstration 
counties, the five states participating in the demonstration were more selective in 
enrolling work registrants than the nation as a whole. 


The rate of participation by exempt volunteers was highest in Missouri’s JET 
program, in which 53 percent of the participants during FY 1995 were exempt work 
registrants. Texas and Georgia have also enrolled substantial numbers of exempt work 
registrants as volunteers, reflecting the willingness of these states to support 
participants in self-initiated courses of study. Exempt volunteers made up a much 
lower proportion of the participant caseload in Hawaii and were present only in very 
small numbers in South Dakota. 


Hawaii, Missouri, and Texas also encouraged participation by “mandatory 
volunteers” —mandatory work registrants who volunteered to participate in the 
demonstration before being called in for services. In Hawaii and Missouri, which kept 
track of participation by mandatory volunteers, these individuals comprised 57 percent 
and 8 percent of the demonstration participant caseload, respectively. It was not clear 
how many participants were mandatory volunteers in Texas, where this status was not 
officially tracked. 


9 State aggregate statistics prepared by South Dakota and submitted to the Department of 
Agriculture showed that the number of “component placements” was 20 percent of the mandatory work 
registrant pool for FY 1994. Although the state number double counts participants in more than one 
component, it is not clear why this is so different from the 10 percent computed from the evaluation’s 
client-level database. 
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In the four states that designated priority target groups, preliminary statistics 
provided by the states in their process and implementation evaluations show that the 
demonstrations reached a broad mix of target and non-target group members. The 
greatest focus on target group members occurred in Georgia, where around 81 percent 
of all demonstration participants belonged to one or more of the designated target 
groups. This pattern resulted from the fact that, in Georgia, all target group members 
were Called in before any non-target group members were called in. 


In the remaining states, target group representatives constituted at least 50 percent 
of all demonstration participants, but a number of non-target group members 
participated as well. Non-target group members were recruited and enrolled either as: 
(1) volunteers, who were given priority over all other groups in most states; (2) after 
E&T staff had called in all available target group members referred in a given month 
(in Georgia and South Dakota); or (3) as part of a pre-determined mix of target and 
non-target group work registrants (in Hawaii). 


Characteristics of demonstration participants varied widely from state to state, 
reflecting variations in the E&T work registrant pool (e.g., in the proportion of single 
individuals versus individuals from family households and males versus females) as 
well as variations in the types of individuals that were targeted by the demonstrations 
and participated in demonstration services. In general, the state process evaluation 
reports did not include much information on shifts in the demographic characteristics of 
E&T participants as a result of the demonstration. !° 


Several states noted a significant increase in the proportion of female participants 
between the demonstration and non-demonstration (or pre-demonstration) sites/periods. 
For example, in Missouri, E&T participants in non-demonstration counties were 51 
percent female, while demonstration participants were 82 percent female. In South 
Dakota, females increased as a proportion of all E&T participants from 41 percent prior 
to the demonstration to 62 percent during the first full year of demonstration 
operations. Although the process reports for Texas and Georgia did not provide 
comparisons of gender between the demonstration and non-demonstration counties, 





10 Several states compared the education levels and household composition of demonstration 
participants to the characteristics of all Food Stamp work registrants or all Food Stamp recipients, but 
were unable to provide information about the characteristics of E&T participants prior to the 
demonstration or in non-demonstration sites. 
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both states noted that females were heavily over-represented in the volunteer caseload 
within the demonstration counties. Several factors may have interacted to contribute to 
increased participation by females in these states, including: the inclusion of women 
with children under 6 in the mandatory work registrant pool under JOBS rules, 
increased reimbursements for child care expenses, requirements for mandatory 
participation by caretakers of young children if they were under 20 and had not 
completed high school, and the ability of volunteers to access supportive services 
including child care while participating in self-initiated education and training. 


Changes in Sanctioning Criteria and Procedures 


Each of the demonstration states was encouraged to implement the JOBS 
sanctioning rules as part of the conformance demonstration. This was perceived as 
“giving more teeth” to the E&T sanctioning policies by increasing the duration of 
sanctions for the second and third failure to cooperate from two months to three and six 
months, respectively, and requiring participants to wait until the end of the designated 
sanction period before being able to “cure” the sanction. (Under E&T rules, an 
individual may cure a sanction at any time.) It was hoped that tougher sanctioning 
policies and procedures would decrease the “no-show” rate among mandatory E&T 
work registrants and increase the program participation rate among individuals called in 
for services. !! 


Tougher sanctions were implemented, as planned, in Georgia, South Dakota, and 
Texas. However, in Hawaii, because of historically lenient conciliation procedures for 
sanctioning under JOBS, the conformance with JOBS sanctioning policy was perceived 
as weakening rather than strengthening existing sanctioning procedures. !2 
Furthermore, in Missouri, as the result of an administrative oversight, sanctioning 
procedures were not applied by demonstration case managers to any demonstration 
participants. When state administrators were questioned about this practice, they found 
that case managers had been reluctant to administer sanctions because of their efforts to 


11 As a result of conforming the work registration requirements for JOBS and E&T, participating 
states were also able to coordinate the sanctions applied to JOBS participants, so that both Food Stamp 
and AFDC benefits were affected when individuals failed to comply with JOBS participation 
requirements. This, in turn, was perceived as increased the effectiveness of sanctioning within the JOBS 
program. 


12 JOBS sanctioning procedures in Hawaii were amended during FY 1995 to make the conciliation 
process less drawn-out. 
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encourage participation among mandatory and exempt volunteers. After the lack of 
sanctions was brought to the attention of Missouri state administrators in the spring of 
1996, they indicated that the conciliation and sanctioning procedures described in the 
JOBS operations manual would be implemented for the rest of the demonstration 
period. 


Exhibit II-5 summarizes the preliminary information provided by the 
demonstration states on how the demonstration affected noncompliance and sanctioning 
patterns. Because of the differences in client targeting policies and call-in practices 
between the demonstration and non-demonstration periods/sites, it is misleading to 
directly compare the numbers of adverse action notices issued to the size of the 
mandatory work registrant pool. However, in the absence of data on the number of 
individuals called in for services in each state, Exhibit III-5 suggests some interesting 
patterns. First, it suggests that the demonstrations that continued to call in relatively 
high proportions of mandatory work registrants (e.g., South Dakota) continued to 
experience high “no-show” rates and issue high rates of adverse action notices. 


Second, Exhibit II-5 suggests that states like Missouri and Hawaii, which 
emphasized recruitment of exempt and mandatory volunteers, may have experienced 
reduced noncompliance rates during the demonstration compared to the rates during 
their respective non-demonstration sites or periods. These reduced rates of 
noncompliance could have resulted from lower call-in rates, higher response rates 
among work registrants called in, and/or higher ongoing compliance rates among 
individuals enrolled in program services. In Georgia, once individuals were selected 
from the referral pool and entered into the E&T MIS system, they were more likely to 
comply with participation requirements in the demonstration counties (which offered 
more individualized case management and enhanced supportive services) than in the 
non-demonstration counties. As shown in Exhibit III-5, Georgia’s four demonstration 
counties experienced a 23 percent noncompliance rate among individuals who were 
selected, which was substantially lower than the 41 percent noncompliance rate 
experienced in the non-demonstration counties. 


SERVICES AND SERVICE PLANNING 


All of the states implemented the demonstration with the intention of increasing 
the range and intensity of education and training services offered and more carefully 
matching these services to the individual participant’s needs and interests. 
Simultaneously, most were also revamping their regular E&T programs with similar 
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Exhibit ITI-5 
Shifts in Sanctioning Rates 


Demonstration State | Percentage of Work | Percentage of Work | Difference Between 
Registrants Issued Registrants Issued | Demonstration and 
Notices of Adverse | Notices of Adverse | Non-Demonstration 
Action in Action in Non- in Percentage Points 
Demonstration Site Demonstration 
Site/Period 


Georgia 41% 
4 counties 


Hawaii 19.9%8 
1 county 


Missouri Not included in Decrease; no 

8 counties process report sanctions requested 
for demonstration 
participants as of 


South Dakota FFY 1994: 46% FFY 1992: 49% 
20 counties 


Texas Not included in Not included in 
1 county process report process report 





*Among terminees screened between 7/1/93 and 6/30/95. 
tAmong terminees screened between 7/1/93 and 6/30/95. 
+Among work registrants referred to E&T between 11/93 - 6/95 


SAmong work registrants referred to E&T between 11/93 - 6/95 
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objectives in mind. The process studies conducted by the various states, including their 
comparisons of demonstration and non-demonstration service designs, offer interesting 
findings about how the demonstrations influenced service options and resulting service 
utilization patterns. 


E&T Services Prior to the Demonstrations 

Prior to FY 1993, the E&T programs in all of the states heavily emphasized job 
search activities with or without job readiness training. This pattern was most striking 
in South Dakota, Georgia, and Texas. In South Dakota, job search and job readiness 
training were the only two recognized activities. In Georgia, virtually all E&T clients 
participated in group job search followed by individual job search. A few individuals 
who expressed an interest in education or training were permitted to enter these 
optional components, but only after they had completed group job search. In Texas, 
brochures describing education and training offered by JTPA and the Texas Education 
Agency (which provided adult basic education and GED programs) were made available 
to participants, but, as in Georgia, referrals were almost never made in practice. 
Instead, the contractor for E&T services enrolled most participants in directed job 
search, which was supplemented in some counties by an optional job search training 
component, consisting of a 20-hour regimen of job counseling and group job search. !3 


Although still emphasizing job search and job readiness activities, Hawaii and 
Missouri offered somewhat more varied E&T service options prior to FY 1993. In 
contrast to Georgia and Texas, where participants could undertake education and 
training only if they took the initiative in expressing an interest in these activities, E&T 
employment counselors in Hawaii provided selected participants with referrals to 
training activities, primarily to basic education programs or work experience. 

However, the majority of E&T clients were still assigned to supervised job search, with 
some also receiving an initial 24 hours of instruction in job search techniques. 


Missouri followed a somewhat different approach to making education or training 
services available to E&T participants. This state’s Division of Family Services (DFS) 
transferred a portion of E&T funds to the Division of Job Development and Training 





13 During this period, Texas also conducted a periodic data match of Food Stamp and JTPA data 
files. Food Stamp recipients who turned out to be JTPA participants were considered to be enrolled in 
the E&T program as exempt volunteers; many in fact never knew they were E&T participants, although 
they appeared as such for reporting purposes. 
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(DJDT), which oversaw the state’s JTPA program. The DJDT in turn distributed these 
funds to selected JTPA service delivery areas for the enrollment of individuals 
interested in JTPA services who were receiving Food Stamps but not AFDC. These 
individuals were reported back to DFS and classified as E&T participants. The E&T 
program’s financial agreement with JTPA reflected the state’s intention to broaden 
training opportunities for Food Stamp work registrants, although it is not clear whether 
this arrangement resulted in the recruitment or enrollment of significant numbers of 
individuals who would otherwise have been unlikely or unable to access JTPA services 
on their own. Meanwhile, the majority of E&T participants were served by the 
Division of Employment Security (DES), which required participants to undertake 
supervised job search. 


Presumably because service sequences were largely predetermined in the E&T 
programs in most states, assessment and service planning were typically given cursory 
treatment. In South Dakota, for example, individualized service planning was largely 
irrelevant, given that all participants were expected to participate in the same activities. 
In Missouri, Georgia, and Texas, much the same could be said for the majority of E&T 
participants, who were channeled into job search and/or job search training 
components. However, participants who self-initiated or specifically requested training 
in JTPA or adult education programs in these states received the formal objective 
assessment and service planning that these programs routinely offered. Individual 
education or training plans were required for E&T participants who entered education 
services in Missouri and education or training services in Georgia. 


Hawaii seems to have offered more extensive individualized assessment and 
service planning. Presumably this is related to the fact that it referred an appreciable 
number of its E&T clients to education and training services, as discussed above. In its 
regular E&T program, employment counselors routinely collected information on 
participants’ work history, vocational skills and interests, educational competencies, 
and physical capabilities and limitations, and would administer interests and/or 
proficiency tests, if appropriate. Job-related counseling might be provided to assist 
participants in making realistic vocational choices, exploring alternative fields of work, 
and understanding personal attributes or barriers that prevented them for getting or 
keeping a job. Based on these results, an employability plan was developed for each 
participant that set forth career goals and a planned series of actions for attaining those 
goals, identifying specific services and training components to be undertaken and 
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specifying tentative time frames for implementing each step of the plan. As changes 
occurred, the plan was to be revised. In actuality, most participants in Hawaii’s E&T 
program during the early 1990s were placed in a narrow range of service activities, but 
attention was at least being paid to assessment and service planning as formal 
processes. 


Demonstration Service Designs 


The conformance demonstrations expanded the range and flexibility of service 
offerings in every state. They also gave greater attention to individualized service 
planning. However, the degree of change should not be overstated. Indeed, all states 
continued to make heavy use of job search and/or job readiness training components 
and the majority of demonstration clients in most states continued to participate 
exclusively in job search or job search training. However, most demonstrations 
increased the range and intensity of service options available to E&T participants by: 
(1) redesigning job search and job search training activities: (2) improving participant 
access to education and training services through direct purchase arrangements and/or 
non-financial referral linkages; (3) providing individualized assessment and service 
planning and intensified case management; and 4) offering a wider range and more 
generous level of supportive services. 


One notable difference in demonstration services, compared to previous designs, 
was the addition of greater richness to the job training and job search components. 
For example, Texas expanded the duration (from 20 to 80 hours) and broadened the 
focus of job readiness and life skills workshops required as a prelude to job search. It 
also increased the number of required job search contacts and instituted a group job 
search seminar, in which participants undertaking job search were required to meet as a 
group for a few hours each week to share job leads and discuss job search skills. 
Similarly, Georgia increased both the intensity of supervision and the required number 
of job search contacts for those undergoing job search. Hawaii introduced a 6-week 
pre-employment curriculum designed to foster self-esteem as an option for participants 
assessed as needing this service. 


More significantly, all demonstration states moved away from standardized 
service sequences and improved clients’ access to education or training at least to some 
degree. Without the expectation that all clients would necessarily receive job search 
assistance and/or job readiness training, case workers began to consider alternative 
Service strategies. Several states improved access to these alternatives by directly 
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purchasing education or vocational training slots for participants. For example, 
Missouri and Hawaii would pay tuition for education or vocational skills training, when 
necessary, to ensure that access to the training regimen recommended by the case 
manager could be obtained. Similarly, Georgia used some of its cooperative agreement 
funds to establish an in-house adult basic education program in 1 of the 4 demonstration 
counties, which offered training of somewhat greater intensity than was provided by the 
state education agency. 


Demonstration states also improved access to education and training by fostering 
closer interagency collaboration and strengthening non-financial agreements with 
education and training providers. Texas, for example, pointed out in its process report 
that local interagency collaboration was much stronger in BOND, its demonstration 
project, than was the case in its E&T program, because agency rules and 
responsibilities in BOND were supported by legislative mandates, explicit interagency 
agreements, and periodic local interagency planning conferences or meetings. Asa 
result of BOND, Texas strengthened non-financial agreements with the local adult 
education agency and post-secondary educational institutions. Consequently, many 
demonstration participants found their way into adult education or post-secondary 
education programs. The availability of taxpayer-funded educational offerings, tuition 
assistance from Pell grants or other student financial aid programs, and supportive 
services provided by the demonstrations proved to be a potent combination in 
encouraging increased educational enrollments among demonstration participants not 
only in Texas, but also in Georgia. 


Several states found that non-financial linkages were more difficult to cultivate 
with the JTPA system than with the educational system, perhaps because of declining 
overall JTPA resources. At the outset of the Texas BOND demonstration, use of JTPA 
resources to support vocational training by BOND participants was relatively rare. 
However, by the end of the demonstration period, “dual enrollment” was more 
common (often initiated through “reverse referrals” made by the JT[PA system to obtain 
BOND supportive services for JTPA applicants) and BOND case managers were 
outstationed at the JTPA service provider on a regular basis to facilitate dual 
enrollment. In Missouri, the provision of demonstration funding to purchase services 
from the JTPA system on behalf of demonstration participants was key to securing the 
enthusiastic cooperation of JTPA service delivery areas as JET service providers. 
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The movement away from standardized service sequences was associated with 
more thorough assessment, individualized service planning, and intensified case 
management and constituted a critical adjunct to supporting the expanded service 
options in several states. All states moved to implement individualized service planning 
and case management as part of their demonstrations to at least some degree, and 
several states reported that these elements represented integral parts of the 
demonstration service designs. In Georgia and Missouri, for example, demonstration 
case managers conducted detailed assessments and developed individualized training 
plans for all participants. In-person meetings with case managers to check on the 
participants’ progress occurred periodically and participants were encouraged to make 
frequent telephone contacts with their case managers to report on their progress, 
request additional services, and share successes and disappointments. Oversight was to 
continue in the post-placement period for clients who had obtained employment. Texas 
demonstration staff also conducted individualized service planning and case 
management, although typically just for participants who were categorized as less job 
ready. These participants received a service needs assessment, followed by the 
development of an individual service strategy. 


In Hawaii’s PRIDE program, case management and individualized service 
planning also were given substantial emphasis, but took a somewhat different form. In 
this state, substantial attention was devoted to identifying and redressing “barriers” to 
employment, as part of the service planning process, including those relating to health, 
psycho-social characteristics, financial capabilities, family circumstances, and education 
and skills. 


By contrast, service planning was less well developed in South Dakota’s statewide 
demonstration, and case management was not recognized as an official program feature. 
Nonetheless, even in this state, attention to participants’ individual needs were given 
somewhat greater attention than in the E&T program prior to the beginning of the 
demonstration. 


Another important change associated with the demonstrations in all states was the 
enhancement of supportive services available to participants. As part of the 
conformance demonstration, all states revised their E&T supportive service policies to 
be more similar to their JOBS policies. These changes typically allowed for more 
generous payments for transportation assistance and work and incidental expenses, and, 
in some states, for more generous demonstration-funded child care reimbursements. 
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In contrast to the E&T limit of $25 per month for transportation and other 
training-related expenses, the maximum transportation allotments rose appreciably 
under the demonstrations in all states—for example, up to $230 per month in Texas, 
$70 per week in Missouri, and $3 per day in Georgia. Policies regarding payments for 
work and incidental expenses, which were part of the $25 monthly limit under E&T 
regulations, also increased substantially under the demonstration. In accordance with 
its JOBS supportive service policy, South Dakota offered reimbursement for minor auto 
repairs or for clothing or tools under the demonstration. Missouri and Texas could pay 
for one-time work-related auto repairs, tools, uniforms, and miscellaneous incidentals 
in their demonstration programs. Hawaii and Georgia were most generous in the range 
and level of supportive service payments they would authorize—payments for work and 
incidental expenses including eye wear, work clothing, certificates and licenses, testing 
fees, union dues, and medical and dental expenses were all supportive services that 
could be authorized by case managers on behalf of their clients under these states’ 
demonstrations. 


While the demonstrations’ more generous transportation and incidental payment 
schedules doubtless facilitated program participation for many individuals, it was the 
effects of expanded child care reimbursements that were the most remarkable, at least 
in two of the three demonstration states that offered child care assistance based on the 
JOBS model. Child care allowances were typically used only infrequently in the non- 
demonstration E&T programs in most states, because those with responsibility for the 
care of young children were usually exempted from program participation. Following 
the JOBS model, however, work registration rules no longer automatically exempted 
those with young children. As a consequence, some states—especially Georgia and 
Texas—found that the demand for child care assistance skyrocketed. 


Texas, Georgia, and Hawaii each offered child care reimbursements at the JOBS 
level in their demonstrations.!4 Both Georgia and Texas found that their demonstration 
expenditures for child care far exceeded budget expectations. In Georgia, a substantial 
proportion of cooperative agreement funds were used for this purpose. In Texas, child 
care expenditures, funded from the statewide E&T budget, exceeded $500,000 during 


14 Georgia offered child care assistance at the JOBS level throughout its statewide E&T program, 
using 100 percent state funds to make up the difference for the non-demonstration E&T program 
participants. 
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the first two years of the demonstration. Although the level of child care expenditures 
increased in Texas as a result of the demonstration, it did not appear to be merely the 
increased level of child care payments available that increased the demand for this 
payment. Rather, the increased level of child care expenditures resulted from the 
interaction among several factors, including: revised exemption policies that included 
more parents with young children as mandatory work registrants, an enhanced menu of 
service options, policies encouraging participation by volunteers, and the availability of 
expanded child care assistance. Both states in fact described the child care assistance 
they provided as “magnets,” prompting caregivers responsible for young children to 
volunteer for program participation. In Texas, these were typically persons interested 
in undertaking post-secondary education. In Georgia, a wider variety of individuals 
volunteered for the program because child care assistance could be used to support 
either training or the first 90 days of employment. 


Curiously, Hawaii, which almost doubled the child care allowance per dependent 
under the demonstration (from $160 per dependent per month in its E&T program to 
$325 per month under the demonstration), did not report that child care was a special 
attraction for participants, nor did child care allotments appear to consume an 
inordinate proportion of demonstration funds. This might be attributed to the fact that a 
smaller proportion of E&T work registrants in Hawaii are from households with 
children. (As a result of Hawaii’s relatively generous AFDC benefits, Food Stamp 
recipients with children are likely to remain eligible for AFDC even if they have low- 
wage employment, and thus, be served through JOBS than through the E&T program.) 


In contrast, neither Missouri nor South Dakota routinely made child care 
reimbursements with demonstration funds. As a matter of policy, South Dakota would 
approve child care reimbursements only on a short-term or temporary basis, referring 
those who needed assistance on an ongoing basis to the state’s social service block grant 
funds set aside for this purpose. Missouri also decided against offering child care 
assistance as part of its demonstration. It found as a consequence that many potential 
participants who were parents needed to be exempted from services because the lack of 
child care precluded their participation. (Very recently, Missouri rethought this policy, 
and began offering child care assistance as part of its demonstration, on a limited 
basis.) 
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Comparison of Demonstration and Non-Demonstration Service 

Designs 

Shortly before demonstration start-up, or while the demonstrations were ongoing, 
several states moved to modify their statewide E&T services designs. These changes 
were facilitated by changes in federal performance-standards requirements. However, 
the process of applying for demonstration funds also prompted several states to rethink 
their statewide E&T operations and refocus their efforts towards selective client 
targeting and more individualized service planning for those enrolled. Thus, FY 1993 
marked the beginning of efforts in some states to move towards individualized service 
planning and broaden the range and intensity of services available to all E&T 
participants statewide. This pattern emerged most clearly in the E&T programs in 
Georgia and, to a lesser degree, in Missouri and Texas. In this section, we review the 
residual differences between demonstration and non-demonstration services. 
Interestingly, as documented in the next section, these efforts typically did not lead to a 
marked expansion of the education and training services actually utilized in the non- 
‘ demonstration sites. 


In an effort to revamp its regular E&T program along the lines of the JOBS 
model, Georgia’s Department of Human Resources relaxed the requirement that 
virtually all participants should be funneled through job search. Instead the state left 
the choice of training regimen up to the E&T case managers in the individual counties. 
In principle, participants could be given a non-financial referral to JTPA or adult 
education programs, so that they could access adult basic education, GED preparation, 
post-secondary or classroom vocational skills training, or on-the-job training— 
essentially, the full range of education and training activities to which demonstration 
participants could be referred, with the exception of any enhanced services paid for 
with demonstration funds. Thus, in theory at least, the primary differences between the 
demonstration and non-demonstration counties consisted of enhanced supportive 
services and the fact that case managers sometimes used demonstration funds to 
purchase training for demonstration participants. In fact, it appears that demonstration 
participants actually received more individualized service planning and case 
management than E&T participants in the non-demonstration counties. Furthermore, 
the provision for participation by volunteers in the demonstration counties resulted in 
an increased level of interest in education and training services among demonstration 


participants. 
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Missouri, which had been funding training for E&T participants who wished to 
enter JTPA even before the demonstration, expanded its agreement with the Department 
of Labor and Industrial Relations’ Division of Job Development and Training in 
FY 1994 to promote JTPA services in a larger number of service delivery areas, and 
began requiring that Food Stamp E&T participants served in JTPA should be provided 
access to the full range of JTPA training services. Thus, in this state too, education 
and training opportunities were expanded to parallel those available in the 
demonstration counties. Additionally, Missouri enhanced the services provided to E&T 
participants by the Department of Industrial Relations’ Division of Employment 
Security, its chief contractor for E&T services, by requiring the introduction of three 
half-day sessions of job search training as a prelude to job search. Despite these 
changes, substantial differences remained between the demonstration and non- 
demonstration E&T operations. In non-demonstration counties, most E&T participants 
received job search training, followed by job search. In contrast, in the JET 
demonstration counties most participants were assisted in pursuing individual education 
or training plans. 


Texas represents a middle ground between Georgia—which made a serious effort 
to offer a similar menu of services to all E&T participants statewide—and Missouri— 
which operated substantially different service designs in demonstration and non- 
demonstration counties. For participants assigned to employment preparation and job 
search services, Texas substantially closed the gap between demonstration and non- 
demonstration services during FY 1994 and FY 1995. In FY 1994, the state 
consolidated its service delivery arrangements for E&T and JOBS services statewide. 
As part of this process, E&T components provided by the Texas Employment 
Commission were redesigned to emphasize intensified job readiness training and a 
greater attention to life skills development, making these components essentially 
identical to the ones offered in the demonstration county. Additionally, as in the 
demonstration county, the group employment seminar was instituted for participants 
undertaking job search. However, the demonstration county remained substantially 
different from the regular E&T program in the services provided to less job-ready 
participants. For these participants, the greater range of education and training services 
used and the enhanced case management and supportive services available to support 
participation in more intensive services distinguished the demonstration service design. 
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In South Dakota, the E&T conformance demonstration was implemented in all 
counties where E&T services were provided, so it makes no sense to talk about the 
evolution of E&T services apart from the onset of the demonstration. In South Dakota, 
the key differences between the demonstration and the previous E&T service design 
included a slightly more individualized service planning process, a wider range of 
service options based on the JOBS program, and access to an enhanced set of 
transportation and work/training expense reimbursements. 


Hawaii’s non-demonstration E&T program design changed little during the years 
the demonstration was operating on Oahu until FY 1995, when state funding was 
curtailed and “work first” requirements were imposed for participants. The 
demonstration design in Hawaii was distinguished not so much by the menu of 
employment and training services offered to participants as by the introduction of a 
barrier assessment and removal process prior to referral to employment services, as 
well as by an emphasis on the recruitment of volunteers for participation in 
demonstration services. 


INFLUENCE OF THE DEMONSTRATION ON SERVICE UTILIZATION 
PATTERNS 


In their process reports, all the demonstration states reported statistics on the 
utilization of different services both in demonstration and comparison counties. 
However, the statistics provided by the different states differ in the classification 
systems used to describe E&T activities, the time periods covered, whether the 
summaries were based on a “point-in-time” view (e.g., of services used during a given 
month) or a “longitudinal” view (e.g., of services used over a year or multi-year 
period), whether they referred to all participants or just to project terminees, and 
whether an accurate unduplicated count of participants during the reference period was 
available. Because of these differences, the specific statistics provided by the states 
should not be used in direct cross-state comparisons. !° 


In this report, we have used the statistics provided by the states cautiously, as 
rough gauges of the priority given to different services in each state’s demonstration 
experience. In descriptions of each state’s experience, we refer to specific statistics on 
service utilization, as provided by the state process evaluations. However, in summary 





15 We hope that the state impact evaluation reports will provide improved information on 
participation patterns and service usage in a format that will facilitate cross-state comparisons. 
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tables, we have limited our comparisons to describing whether each state experienced 
an increase or decrease in its dependency on a particular type of services between the 
demonstration and non-demonstration period/site, and whether the shift was major (an 
increase or decrease of greater than 10 percentage points in the proportion of 
participants using a service), minor (an increase or decrease of between 5 and 10 
percentage points), or insubstantial (less than a 5 percentage point increase or 
decrease). 


As documented in the individual state process evaluation reports, despite the 
efforts to expand the range of services available to E&T participants in non- 
demonstration counties in Georgia, Missouri, and Texas, overwhelming proportions of 
E&T participants in non-demonstration counties continued to receive exclusively job 
search and job readiness services. In Georgia’s comparison counties, for example, 
almost all participants continued to receive only individual or group job search, despite 
the fact that the state no longer mandated a required sequence of activities and gave 
case managers the discretion to make referrals to JTPA or adult basic education 
programs. Texas continued to report substantial numbers of E&T participants in 
education or vocational skills training throughout the state, but this was almost solely a 
result of after-the-fact data matches with JTPA records. 


Only in Hawaii and Missouri were appreciable numbers of E&T participants 
placed in education and training in non-demonstration counties. In Hawaii, with its 
ongoing emphasis on individualized service planning for E&T participants, about 24 
percent of monthly E&T participants in the comparison county were listed as 
participating in education and training components. In Missouri, the reported rates of 
participation in education and training in the non-demonstration counties were even 
higher, as a consequence of the financial agreements the Department of Social Services 
had in place with the JTPA system. 


Shifts in Service Utilization Under the Demonstration 

Exhibit III-6 summarizes the changes that were associated with the 
demonstrations in each of the states, in comparison with each state’s regular E&T 
activities. These comparisons are between the demonstration and comparison counties 
in all states but South Dakota and Texas, where figures are based on pre/post 
comparisons. As the exhibit shows, in Missouri, Texas, and, to a lesser degree, 
Georgia, the demonstration service designs resulted in pronounced changes in service 
utilization patterns. In Missouri, utilization of education and training activities 
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increased overall, while use of job readiness and job search components was reduced. 
This reported shift in service utilization is consistent with the changes in service design 
described elsewhere in this report, including the shift from reliance on job search 
assistance provided by state Employment Security staff toward individualized services 
planned and supported by case managers hired specifically for the demonstration 
program. To ensure that access to the recommended training regimen could be 
obtained, demonstration funds were used to pay for tuition when necessary. In its 
process report, Missouri indicated that substantial numbers of JET participants attended 
high school or received GED or adult basic education instruction (27 percent) or 
attended post-secondary education (23 percent), in contrast to the comparison counties, 
in which no one was reported as having received these services. However, the 
proportion of demonstration participants receiving vocational skills training (21 percent) 
was slightly lower than the proportion reported in vocational training in the comparison 
counties (34 percent), because of the emphasis on the Department of Social Service’s 
financial agreement with JTPA to secure expanded services for E&T participants in the 
non-demonstration counties. Demonstration participants also were enrolled in on-the- 
job training and work experience at somewhat lower rates than in the comparison 
counties. !6 


In Texas, as noted above, the demonstration service design was distinguished by 
individualized service planning for a substantial subset of demonstration participants— 
primarily less job-ready work registrants as well as individuals interested in post- 
secondary training. Although most work registrants with a high school degree or 
recent work experience were automatically assigned to job readiness and job search 
activities, case managers conducted an assessment and developed an individualized 
service plan for those considered to be less job ready. Because Texas developed 
particularly strong referral linkages with local educational institutions, increased 
proportions of BOND participants were enrolled in educational activities. For example, 
Texas reported that in the pre-demonstration period only about 6 percent of E&T 
participants in the demonstration county were enrolled in adult education and none were 
enrolled in high school or post-secondary education. Two years later, with the 
demonstration in full-swing, these figures had increased to 13 percent in adult 


16 Figures are for program participants and are taken from Missouri’s Addendum to the Process 
Evaluation of the JOBS-Employment and Training Demonstration Program (JET), p. 17. 
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education, 5 percent in high school, and 44 percent in post-secondary education. !7 
However, because coordination with JTPA did not materialize as hoped, enrollments in 
vocational skills training remained quite low. 


Despite the similarities between the menu of services available in theory to E&T 
participants in demonstration and non-demonstration counties, Georgia’s demonstration 
design was distinguished by individualized service planning and a case management 
approach as well as by the use of some of its cooperative agreement funds to establish 
an in-house adult basic education program to supplement existing community services 
in the largest of its four demonstration counties. One consequence was that the rate of 
enrollment in adult education rose to 23 percent in the demonstration counties, 
compared with just 3 percent in the non-demonstration counties. Additionally, Georgia 
used non-financial referrals to link small numbers of additional demonstration 
participants to community resources for post-secondary education, English language 
training classes, vocational schools, or other job skills training. With the exception of 
adult education, the percentage of demonstration participants embarking on these 
training assignments was modest (about 15 percent in total), but still represents an 
appreciable increase over what was provided to non-demonstration E&T participants. !8 


Changes in patterns of service utilization were somewhat less sweeping in Hawaii 
and South Dakota. Hawaii might be said to have implemented marked changes in its 
service strategy, but these were primarily along the lines of expanding assessment far 
more than in other states and assisting participants in redressing any health and psycho- 
social barriers to employability that were uncovered. Unfortunately, many individuals 
whose barriers were uncovered declined to follow-up with the social service agencies to 
which they were referred. Moreover, the demonstration’s focus on assessment and the 
identification of barriers seemed to divert attention from efforts to broaden education 
and training opportunities for participants. Of course, all along it had emphasized 
education and training as part of its E&T program more than other states, and had 


17 Figures represent annual participation rates and are taken from the Texas Food Stamp 
Employment and Training/JOBS Conformance Demonstration: Process Evaluation Final Report (Table 
5.3). Demonstration figures are for FY 1995, while comparison figures are for the demonstration county 
in the year before the demonstration began. 


18 Figures are for FY 1995 for the demonstration and comparison counties, as reported in 
Georgia’s PEACH JOBS Employment and Training (JET) Project: Implementation and Process Findings. 
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placed appreciable numbers of participants in a variety of education and training 
activities, including basic skills remediation, post-secondary education, work 
experience, and community employment and training. In the demonstration county, the 
mix of service activities changed somewhat (e.g., towards greater use of post-secondary 
education and away from community employment and training), but the percentages in 
all of these activities combined actually fell slightly, compared to the service mix 
reported for the non-demonstration county. 


South Dakota also experienced a more subtle shift toward expanded utilization of 
education and training services. Under its statewide demonstration, South Dakota 
implemented an individualized service planning process. However, as a practical 
matter, this plan seemed to have only limited bearing on the course of services to which 
participants were assigned. Although the menu of available services was greatly 
expanded under the demonstration, job readiness and job search components were still 
emphasized as the favored avenues for assisting the work registrants to enter 
employment. South Dakota’s process report reported that 16 percent of demonstration 
participants were enrolled in post-secondary education, but few participants entered any 
additional service categories, other than job search or job readiness training. !9 


Looking at Exhibit III-6 row-wise (as opposed to column-wise) shows that 
assessment and service planning were emphasized in all states’ demonstrations, 
compared with their E&T operations. This can be taken as clear evidence that states 
were attempting to move away from standardized service sequences and towards the 
JOBS model of individualized employability development. 


Educational activities also increased almost uniformly, including post-secondary 
education (in all states but South Dakota), adult education (in all states but Hawaii and 
South Dakota), and high school (in all states but Georgia and Hawaii). Thus, the JOBS 
program’s emphasis on educational development, particularly for young people without 
a high school education, appeared to be realized in the demonstrations. 





19 Figures are taken from South Dakota Food Stamp E&T Conformance Demonstration: A 
Combined Implementation and Process Report and are for FY 1994 for the demonstration period and 
FY 1991 for the comparison. South Dakota, which was running its demonstration statewide, announced 
plans to substantially expand its use of on-the-job training in its E&T program beginning in FY 1996, by 
directly funding OJT training slots. 
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States were less successful in boosting opportunities for vocational skills training 
and on-the-job training. In fact, no state reported an appreciable increase in either of 
these activities. Similarly, work experience showed no notable increase anywhere 
except in Texas, and its incidence actually declined in Hawaii. 


Meanwhile, the frequency with which individual or group job search activities 
was used declined everywhere, usually substantially so, and job readiness training 
declined in all states but Texas. 


Finally, in keeping with conformance, supportive services where everywhere 
made more generous, although Missouri and South Dakota typically did not provide 
child care allowances. 


Factors Explaining Increased Utilization of Education and 

Training Services in the Demonstration 

As Exhibit III-6 showed, all states moved to implement or expand their use of 
assessment and individualized service planning, and almost all the demonstration states 
appear to have been successful in increasing the range of training options available to 
and used by participants. In several states, regular E&T programs were also evolving 
along similar lines, as described above, but, in practice, the demonstrations’ efforts in 
most states represented a qualitative step forward, away from the one-size-fits-all 
approach and towards individualized services. 


Curiously, rates of service in education and training components increased in the 
demonstration even though E&T participants in non-demonstration counties in all states 
but South Dakota were nominally entitled to receive virtually the same services as 
demonstration participants. Georgia and Missouri made a point of emphasizing this 
fact in their process reports, proclaiming that the service options available to 
participants had changed very little under the demonstration. The fact that service 
usage did in fact change, and did so in some states quite dramatically, is an important 
outcome. 


Several factors in particular seem to be important in explaining the increased 
usage of education and training services. First, voluntary participation was a prominent 
fact of the demonstrations in all states but South Dakota, as a preceding section of this 
chapter has reported. Compared with mandatory work registrants, volunteers 
potentially consist of many more persons who are actively seeking employment and 
training opportunities. By relying heavily on volunteers for their case loads, the 
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demonstrations by this fact alone might have increased the demand for education and 
training, even if nothing else had changed. The fact that, South Dakota and Hawaii, 
the two states with the lowest rate of volunteers among demonstration participants (see 
Exhibit III-3, above) also showed the smallest increases in participation in education 
and training lends credence to this relationship. 


Second, in conformance with JOBS, the demonstrations’ regulations made referral 
to educational programs mandatory for young people without a high school diploma or 
the equivalent. Thus, not surprisingly, the incidence of high school education and adult 
basic education/GED training rose fairly consistently as part of the demonstration 
experience (see Exhibit III-6). 


Third, the heightened attention most demonstrations gave to case management and 
individualized service planning also seems important. As part of the process of moving 
from a one-size-fits-all approach, several states implemented fairly extensive upfront 
assessment, with the assessment results used to identify a service plan that would 
increase the participant’s employability. Given that many states also targeted their 
services towards harder-to-serve work registrants as part of their demonstration, 
education and training activities were called for in many cases. Thus, Georgia and 
Missouri, two of the states that moved most decisively towards individualized service 
planning and case management in their demonstration designs, were most effective in 
expanding opportunities across a range of education and training categories. 


Fourth, rates of participation in education and training offerings typically 
increased the most where the demonstrations had forged the strongest linkages with 
training providers. For example, Texas developed particularly strong referral linkages 
with adult education and secondary education providers in the demonstration county. 
Consequently, many demonstration participants found their way into adult education or 
post-secondary education programs. By contrast, Texas did not develop strong non- 
financial referral linkages with the JTPA program serving the demonstration county 
until late in the demonstration period, and low JTPA service levels to demonstration 
participants resulted. 


Linkages that were supported by financial agreements were, not surprisingly, 
even more effective. Thus, as part of its demonstration, Missouri paid tuition for its 
participants in educational or other training programs when the training opportunity 
could not be accessed otherwise. As a consequence, it could boast of increased service 
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to participants across a range of educational and training categories, including adult 
basic education, post-secondary education, and vocational training. Similarly, Georgia 
used demonstration funds to support an in-house adult basic education program, and as 
a consequence its rates of service to demonstration participants in this component were 
substantially greater than for any other of its educational or training activities. Indeed, 
it complained in its process report about the lack of available training slots in JTPA. 


Finally, the more generous supportive services provided by the demonstrations in 
combination with the inclusion of new groups of work registrants (e.g., UI recipients 
and families with young children) doubtless played a role in making education and 
training feasible for participants. As mentioned, child-care allotments were a 
particularly powerful magnet and absorbed substantial proportions of the 
demonstrations’ budgets in Georgia and Texas. Doubtless some individuals would have 
been unable to support themselves through training of any appreciable duration without 
the child care and other assistance that was provided. 


SUMMARY 


Although the five states participating in the E&T/JOBS conformance 
demonstration shared overall project goals and used the JOBS model as the framework 
for redesigning their E&T programs, there were significant variations across states in 
demonstration policies and practices. 


Individual assessment, service planning, and case management were key elements 
of the demonstration designs in all states. As a result, the number of work registrants 
participating in each demonstration was constrained by the ability of case managers to 
schedule individual intake sessions and manage their assigned caseloads. Despite 
variations in client targeting policies and call-in procedures, each demonstration 
enrolled only a relatively small proportion, ranging from 1 percent to 10 percent, of the 
mandatory work registrant pool. 


Four of the five demonstration states developed policies of giving priority to 
“high risk” target groups. Members of target groups made up over half of the 
demonstration participants in each state, but most states also served non-target group 
members. Four of the five states also gave priority to participation by volunteers. Asa 
result of the active recruitment of volunteers, volunteers made up more than 60 percent 
of the demonstration participants in two states and between 30 percent and 40 percent 
of all participants in two additional states. 
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Although it was hoped that the tougher sanctioning policies associated with JOBS 
would increase participation rates under the demonstrations, this was not the case in all 
demonstration states. In general, the states with lower call-in rates and higher 
participation by volunteers issued notices of adverse action at a lower rate under the 
demonstration than previously, while the states that called in high proportions of the 
mandatory work registrant pool continued to experience high “no show” rates. 


In comparison with previous E&T services, demonstration service designs were 
distinguished by (1) intensified job training and job search components; (2) greater 
flexibility of service sequences and improved access to education and vocational 
training opportunities; (3) more thorough assessment, individualized service planning 
and intensified case management; and (4) availability of enhanced supportive services. 
All the demonstration states appear to have been successful in increasing the range of 
training options available to and used by participants. In several states, regular E&T 
programs were also evolving along similar lines, as described above, but, in practice, 
the demonstrations’ efforts in most states represented a qualitative step forward, away 
from the one-size-fits-all approach and towards individualized services. 


Among demonstration participants, enrollment in educational activities—including 
adult education, post-secondary education, and high school—increased in most states. 
Thus, the JOBS program’s objective of promoting educational development, 
particularly for young people without a high school education, appeared to be realized 
in the demonstrations. States appeared to be less successful in boosting opportunities 
for vocational skills training and on-the-job training. The state impact evaluations 
currently underway should provide additional information on how the demonstrations 
affected participation and service utilization patterns, controlling for differences in 
client characteristics, as well as addressing how the demonstration influenced 
participant outcomes. 
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IV CONCLUSIONS 


THE FEASIBILITY OF CONFORMING E&T AND JOBS 


The experiences of the demonstration states suggest that conformance between the 
E&T and JOBS programs at the state and local levels is administratively feasible. 
Moreover, in each demonstration state, conformance of the work registration, 
exemption, and sanctioning rules and procedures between E&T and JOBS was 
perceived as a significant improvement over the operation of two welfare-to-work 
programs with different rules and regulations. In conforming their E&T programs to 
the JOBS model, Georgia and Texas also successfully tested full consolidation of the 
two programs using integrated service delivery units and, in the case of Texas, 
integrated case management caseloads. 


However, administrative simplicity, by itself, is not a sufficient reason to adopt 
the JOBS model for the E&T program. Beyond the reductions in bureaucratic 
complexity, a number of the programmatic changes arising out of the conformance of 
E&T to the JOBS model made sense to program policymakers and direct service staff. 
Program changes that were generally felt to be beneficial for E&T clients included: 


e The replacement of a “one size fits all” sequence of services with a 
more diverse menu of services and individualized service planning. 


e The redesign of the content of job readiness/job search assistance 
components to help improve participants’ self-knowledge, self-esteem, 
and job search skills. 


e The encouragement of participants with limited educational skills to 
attend basic skills training and complete their high school diplomas. 


e The availability of an enhanced menu of supportive services, including a 
higher rate for reimbursements of child care and transportation 
expenses. 


An assessment of whether the services provided to participants under the 
conformance demonstrations led to improved outcomes is currently being performed by 
the states in their impact evaluations and will be reported in the Synthesis of Cost and 
Impact Evaluations. Without information about how the demonstrations affected 
participant outcomes, it is difficult to determine whether the demonstrations 
accomplished their objectives. 
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Nevertheless, it is clear from the implementation and process study findings that 
the demonstrations did result in increased participation in education and vocational 
training by some E&T participants as the result of an individualized service planning 
process. This shift towards more intensive education and training services was 
accomplished primarily by linking participants to existing education and training 
resources in the local community, such as JTPA-funded training, adult education 
programs, and programs offered by local post-secondary schools. Three aspects of the 
demonstration designs that were associated with the increased utilization of education 
and training services by demonstration participants included: (1) the provision of 
individualized assessment and counseling by case managers who provided 
encouragement to participants to pursue education or training when needed; (2) the 
availability of enhanced supportive services that could help support participants while 
they were in training; and (3) the priority given to participation by volunteers who were 
already motivated to pursue education or training. 


However, the cost of providing individual case Management and supportive 
services to demonstration participants was significantly higher than under the old E&T 
model, even when most of the actual cost of training was leveraged from other funding 
Streams. As a result, even with the cooperative agreement funds, each of the 
demonstrations found that it could serve fewer participants than previously. Thus, 
selective recruitment and enrollment were also key to the ability of the demonstrations 
to stay within their budgets. 


The five demonstration states illustrate several different approaches to allocating 
available resources among potential program participants and across different program 
activities. Texas and Georgia each designed multiple service tracks that offered 
services to both job-ready and less job-ready work registrants. Both states decided that 
it would be desirable to enroll mandatory and exempt volunteers in self-initiated 
training and support them during training by offering them case management and 
supportive services. Both states also called in significant numbers of mandatory work 
registrants for individual job search counseling (Georgia) or group job readiness/job 
search workshops (Texas). 


In contrast to the multiple track systems in Texas and Georgia, Missouri and 
South Dakota each utilized a unitary service design. Missouri’s demonstration design 
was targeted primarily to hard-to-serve exempt and mandatory volunteers who were 
interested in pursuing further education and training. A very small percentage of the 
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mandatory work registrant caseload was served in this state. South Dakota’s design, on 
the other hand, was targeted almost exclusively to mandatory work registrants. While 
the menu of services was flexible enough to support participation in education and 
training, South Dakota’s program was primarily oriented to encourage immediate 
employment among job-ready clients. 


Hawaii was distinctive among the demonstration states because of its broad family 
focused and multi-disciplinary approach. Like Missouri, Hawaii attempted to reach out 
to the least job-ready individuals among the work registrant pool. However, Hawaii 
attempted to prepare mandatory work registrants for successful labor market 
participation by providing an individualized front-end multi-disciplinary assessment of 
employment barriers, referrals to community health and counseling resources, and 
intensive case management services. The community resources Hawaii tried to 
leverage on behalf of its participants included not only education and vocational training 
resources, but health, mental health, and family support services. After addressing 
some of these personal employment barriers, participants in Hawaii’s E&T 
demonstration were referred to a combination of job readiness and job search support 
services supplemented by basic education when needed. 


LESSONS LEARNED FROM THE CONFORMANCE DEMONSTRATIONS 


The implementation process went smoothly for some states. Other states 
experienced delays in hiring and training demonstration staff or were troubled by staff 
turnover after the demonstration began. Beyond the challenges associated with hiring, 
training, and retaining demonstration staff, the states that appeared to have learned the 
most about how to improve their implementation efforts in the future were the states 
that had attempted to involve several different agencies or levels of E&T agency staff in 
the development and implementation of the demonstration approach. The 
implementation lessons suggested by the experiences of these demonstration states 
include the following: 


e Involve all affected agencies and levels in planning for 
demonstration services. The states that were most successful—both in 
creating cross-agency teams for the recruitment of demonstration 
participants and in delivering a broad range of case management, job 
search, education, and training services—emphasized the importance of 
involving staff from all involved agencies at both the state and local 
levels in demonstration planning. 
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e Cross-train staff to understand how their actions affect the rest of 
the system. Cross-training staff to understand the linkages between the 
actions taken by income maintenance workers, case managers, and 
training providers was also encouraged as a way to present an accurate 
and positive view of demonstration services to participants. Ongoing 
staff training and communication among staff in different positions were 
viewed as critical to good staff morale and the ability of staff to “sell” 
demonstration services to prospective participants, potential training 
providers, and ultimately, to employers. 


Among the most notable lessons learned about demonstration design and 
operations as a result of the experiences of the demonstration states are the following: 


e Take advantage of non-financial coordination linkages to increase 
access by E&T participants to education and training services. 
Although Georgia and Texas did not often use E&T funds to purchase 
education and training services for demonstration participants, they 
succeeded in increasing participant access to more intensive services by 
developing effective referral linkages, primarily with local adult 
education and post-secondary educational institutions. Demonstration 
sites appeared to have more difficulty developing effective coordination 
linkages with JTPA service providers. 


e Recognize that the availability of child care expense reimbursements 
may be a powerful magnet encouraging participation by exempt 
work registrants (“exempt volunteers”) and mandatory work 
registrants who may volunteer to participate prior to being called in 
(sometimes referred to as “mandatory volunteers”). Some 
demonstration states were surprised by the positive response among 
caretakers of young children who were attracted to the demonstration 
program in large numbers by the availability of child care 
reimbursements. Some of these volunteers were “reverse referrals” 
from post-secondary educational institutions who identified the 
demonstrations as an additional source of financial aid for their 
matriculated students; others were individuals who were interested in 
furthering their education but who had not yet begun a course of study. 
The combination of mandatory work registration by women with young 
children and voluntary participation by women interested in child care 
assistance to support their participation in post-secondary education 
generated a high demand for child care services in several states. 


e Do not expect a tougher sanctioning policy, by itself, to increase 
rates of participation or decrease sanctioning rates. Although the 
demonstration states had hoped that the JOBS sanctions for non- 
participation would lead to higher rates of compliance among mandatory 
work registrants, the demonstrations continued to experience high rates 
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of no-shows and failures to comply among those targeted for 
participation. Part of the problem appeared to be that the names 
supplied to the E&T program as mandatory work registrants were often 
off the Food Stamp rolls or exempt by the time they were called in. 
When offered the choice between participation in E&T and tougher 
sanctions, an unmeasured but possibly sizable portion of mandatory 
work registrants chose to leave the Food Stamp rolls or accept sanctions 
rather than participate. 


The experiences of the states participating in the E&T/JOBS Conformance 
Demonstration have to be reevaluated in terms of their applicability to the new world of 
welfare-to-work that was finally enacted in August 1996 by the Personal Responsibility 
and Work Opportunities Reconciliation Act of 1996 (PRWORA). PRWORA replaced 
AFDC and JOBS with state block grants that provide states with considerable design 
flexibility but within a context of stringent work participation requirements that not 
only emphasize “work first” but also place restrictions on how participation in 
education will be counted. PRWORA also places a maximum of five years on the 
receipt of cash assistance under the new program of Temporary Assistance to Needy 
Families (TANF) and limits the Food Stamp eligibility of able-bodied adults between 
the ages of 18 and 50 who do not have dependent children to three months out of a 
three year period unless they are employed 20 hours a week. In this context, states will 
be under tremendous pressure to design services that place large numbers of TANF 
recipients in work activities while reserving education and vocational training for 
limited subgroups, such as teen parent head of households and individuals who 
simultaneously meet the requirement for 20 hours per week of work participation. 


It is not yet clear how the new TANF work requirements and the Food Stamp 
program eligibility limits will affect various states’ decisions on target group strategies 
and service designs for the E&T program. Possible E&T target groups relevant to the 
new PRWORA context include individuals without dependent children who are at risk 
of losing Food Stamp eligibility after three months unless they work at least 20 hours a 
week, individuals who have recently lost eligibility for cash grants under TANF due to 
the expiration of their time limit, as well as individuals at risk of long-term Food Stamp 
recipiency. In developing E&T service approaches, states will have to choose between 
an E&T program that conforms with the primary thrust of their Title IV-A welfare-to- 
work program—i.e., a program that encourages widespread participation in services 
oriented to immediate employment—and an E&T program that offers a variety of 
services selectively targeted and/or individually matched to meet the widely differing 
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circumstances of different subgroups receiving assistance from the Food Stamp 
program. 


On the one hand, the experience of the states participating in the E&T/JOBS 
Conformance Demonstration suggests that state E&T service designs that attempt to 
conform E&T and TANF requirements—where permissible under Department of 
Agriculture regulations for the E&T program—will be administratively feasible. On 
the other hand, while conformance with TANF work requirements and service designs 
may simplify the administration of welfare-to-work systems, states would also do well 
to consider that a number of the conformance demonstration states were able to offer 
more intensive services to E&T participants who were motivated to improve their job- 
related skills through education or training. 


The process evaluations for the E&T/JOBS Conformance Demonstration illustrate 
how the demonstration states addressed difficult E&T client targeting and program 
design choices. As the demonstrations in Texas, Georgia, and Hawaii showed, it is 
feasible to design programs with multiple service tracks that provide large numbers of 
job-ready clients with services oriented to immediate employment as well as smaller 
numbers of “hard-to-employ” individuals with services designed to support 
participation in education and training. Key among the new services offered under the 
E&T/JOBS Conformance Demonstrations were supportive services, which, in 
combination with well-developed coordination linkages, enabled participants to attend 
education or training services available through existing community resources. 


As the only remaining form of welfare-to-work programming that offers an 
uncapped 50 percent match of State expenditures, the E&T program could become an 
increasingly important source of funds to states that are interested in improving the 
employability of hard-to-serve individuals who are ineligible for federally-assisted cash 
assistance to needy families. States’ interests in investing in the future of these 
households may increase over time, as time limits begin impacting TANF eligibility for 
increasing numbers of poor Americans. 


IMPLICATIONS OF THE PROCESS EVALUATION FOR THE COST AND 
IMPACT EVALUATIONS 


Under the terms of their cooperative agreements with the Department of 
Agriculture, the demonstration states are required to submit impact reports, detailing 
the net effects of the demonstrations on service levels and participant outcomes, and 
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cost evaluations, describing expenditures incurred in conforming E&T services. These 
evaluations will be subjected to a cross-site analysis by SPR and findings reported in a 
Synthesis of Impact and Cost Evaluations. The states’ process and implementation 
evaluations set the context for interpreting the findings from these later studies by 
describing the implementation experience and detailing differences in client flows and 
service strategies in demonstration and comparison counties within each state. 


The process and implementation results suggest some important considerations for 
efforts to estimate net impacts as well as important caveats in interpreting subsequent 
findings, including: (1) the difficulty in measuring demonstration effects when both the 
non-demonstration and demonstration programs are in flux; (2) the difficulty of 
comparing demonstration impacts across states; (3) the need to account for an extended 
“transition period” as the pre-demonstration program was being dismantled and the 
demonstration program was initiated; (4) the difficulty of controlling for simultaneous 
transformations of participant characteristics and demonstration services; and (5) the 
potential impacts of the demonstration on participants as well as non-participants. Each 
of these issues is briefly described below. 


Changes in the Non-Demonstration and Demonstration 

Programs Over Time 

One issue stems from the finding documented in the process evaluations that the 
demonstration period was marked by considerable flux in welfare-to-work systems. 
Shortly before the demonstrations began, most of the demonstration states had 
traditional E&T programs that targeted all mandatory work registrants with low-cost, 
low-intensity services. At the same time that they were applying for demonstration 
funds, the majority of the demonstration states were also planning for statewide 
redesigns of their E&T programs to implement many of the features of the JOBS 
service model, including selective enrollment of hard-to-serve participants, enhanced 
supportive services, and the provision of more intensive and more individualized 
sequences of services. Some states, like Georgia, implemented these changes prior to 
the demonstration and had relatively stable demonstration and non-demonstration E&T 
services during the demonstration period. Other states, like Texas and Missouri, 
continued to refine their statewide E&T designs during FY 1993 and FY 1994. 


The fact that non-demonstration E&T services evolved towards the JOBS model 
in several of the demonstration states means that differences between demonstration and 
non-demonstration designs converged substantially beyond what might have been 
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anticipated before the demonstration began. To this degree, differences in actual 
service content were less substantial than they might otherwise have been. Obviously, 
efforts to isolate a significant net effect caused by the demonstration will be more 
difficult to achieve under this circumstance; it may also be the case that differences will 
be smaller in scope. 


An associated problem with estimating impacts when non-demonstration services 
were evolving is that it is problematic to generate a single, stable estimate of the net 
difference caused by the demonstration. A further complication was that the JOBS 
program model itself began to undergo rapid evolution in the face of emerging welfare 
reform initiatives at the state and federal level, especially beginning in FY 1995. As 
these changes were introduced as part of the JOBS program, they were also adopted for 
E&T participants within the conformance demonstration as well. For example, among 
the demonstration states, “Work First” initiatives were introduced in Hawaii, Georgia, 
and Texas during FY 1995 and 1996. As part of these initiatives, a wide variety of 
changes in JOBS service designs and service sequencing occurred, including the 
shortening of allowable education and training periods and an increased emphasis on 
job search and immediate employment for all participants. 


Given that both demonstration and non-demonstration service designs were to 
some degree in flux during the demonstration period, the effect of the demonstration 
must have been changing as well. Estimates of net impact generated by the various 
States thus must generally be viewed as an average over these differences. 
Additionally, because most changes in the JOBS model were implemented quite 
recently, after most states had ceased collecting participant-level data for their impact 
Studies and program-level data for their process studies, we can presume that estimates 
of impacts will ignore states’ most recent demonstration experiences. To this extent, 
the estimation of impacts is for a variant of the demonstration model that, at least in 
several states, no longer exists. 


The Need to Contextualize the Findings on Demonstration 
Impacts 


As documented in the state process evaluations, each state operationalized the 
conformance demonstration through its own unique demonstration design which was 
strongly influenced by the state’s variant of the JOBS program model. In addition, 
each state had its own configuration for the non-demonstration E&T program that was 
operating in the comparison sites/periods. The process evaluation reports prepared by 
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the demonstration states and their independent evaluators did not always include 
complete statistics on the characteristics of participants served or the mix of services 
provided in the demonstration and comparison sites/periods. One of the most important 
functions of the impact evaluation reports will be to provide accurate descriptions of 
participants and services in the demonstration and comparison sites, as well as to 
provide descriptions of demonstration outcomes and analyses of demonstration impacts. 


Furthermore, because of the unique configuration of each demonstration and its 
non-demonstration context, the impact evaluation findings on demonstration effects 
cannot easily be compared across the demonstration states. In interpreting cross-state 
differences in impact and cost findings, it will be important to keep in mind exactly 
what the differences were between the demonstration and non-demonstration 
interventions in each particular state. 


Acknowledging the Effect of the Transition Period 

An additional implication highlighted by the states’ implementation reports is that 
nearly all states experienced a disruption in “normal” E&T program operations during 
the transition from E&T to demonstration services. For example, as part of the process 
of shifting from the old E&T to the demonstration service design in the demonstration 
sites, states curtailed enrollments in the pre-demonstration E&T program during the 
period leading up to the demonstration and, concomitantly, once the demonstration 
began, took a while to build up their demonstration case loads. Impact estimates of the 
effect of the demonstration on participation patterns must take into account these 
patterns and acknowledge that effects might be somewhat different during the period of 
mature pre- and post-demonstration operations than they were during the transitional 
period. 


Controlling for Simultaneous Shifts in Clients Served and 
Services Provided 


An additional feature amply highlighted by the process and implementation 
evaluations is that, under the demonstrations, both client mixes and service designs 
were transformed simultaneously. For example, as part of the conformance 
demonstration, the size and composition of the work registrant pool was changed in 
each demonstration county. Additionally, the procedures used to target and call-in 
work registrants for E&T services differed between the demonstration and non- 
demonstration/pre-demonstration sites as a result of priority group targeting (in 
Missouri, Hawaii, and South Dakota) and recruitment of exempt work registrants and 
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mandatory volunteers (in Texas, Georgia, Missouri, and Hawaii). These changes 
typically gave rise to pronounced compositional differences between the participants 
served (e.g., towards including women with young children or persons who were 
identified as being in a specific target group). These differences must be taken into 
account, along with differences in the intervention provided during the demonstration 
period, in estimating net demonstration impacts. 


For example, differences in employment outcomes between demonstration and 
comparison counties could be due either to the fact that the demonstration services were 
more or less efficacious or to the fact that the people the demonstration served tended to 
have different proclivities for obtaining employment completely apart from the types of 
Services that were delivered. A challenge of the impact evaluations will be separating 
out the effect of differing participant characteristics from the impacts caused by the 
relative effectiveness of demonstration vs. non-demonstration E&T services. 


A particularly difficult challenge in this regard is caused by the fact that 
volunteers make up a substantial proportion of the demonstration caseloads in some 
States (especially Missouri and Texas, but also Georgia and Hawaii), but were virtually 
absent from non-demonstration counties/periods. Because of the mere fact that many 
volunteered for services, demonstration participants must be presumed to be “different” 
on average from participants in the non-demonstration EXT program, where 
volunteering for services is typically not encouraged.! Moreover, these differences 
plausibly relate to proclivities for undertaking education and training and/or to higher 
probabilities of being successful in obtaining employment. Thus, for example, 
volunteers might be more highly motivated, on average, than the (mostly non- 
volunteer) mandatory work registrants served in non-demonstration E&T programs, 
and their greater level of motivation itself may give rise to differences in service usage 
or outcomes completely apart from whatever effect the demonstrations’ service designs 
have had. An important issue for the impact evaluations will be differentiating between 
these effects. Doing so will be particularly difficult because differences in levels of 
motivation across individuals can be presumed to be largely unmeasurable (i.e., they 
cannot be readily controlled for with variables available from client files). 





IThey are also likely to differ from mandatory participants in a number of measurable 
characteristics, such as age, formal education level, and family composition, which may be controlled for 
using multivariate statistical analysis techniques. However, there is no easy way to control for the impact 
of the unmeasured differences such as motivation and initiative. 
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Measuring the Impact on Demonstration Participants Versus 

Non-Participants 

Because of their selection of demonstration participants as the evaluation 
universe, most of the demonstration states will be concentrating on measuring impacts 
for individuals who received demonstration services. While more difficult to measure, 
the demonstrations may also have had important effects on two additional groups: (1) 
individuals who were not called in by the demonstrations who would have been targeted 
for participation under non-demonstration designs; and (2) individuals who were called 
in but who decided to be sanctioned or leave the Food Stamp rolls rather than 
participate. South Dakota is the only state that has decided to measure employment 
outcomes for the broader universe of all mandatory work registrants, rather than only 
for demonstration participants. 
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THE GEORGIA PEACH JET DEMONSTRATION 


GOALS AND OBJECTIVES 


Georgia is operating its Food Stamp E&T/JOBS conformance demonstration in 
four counties, representing a mix of urban and rural areas. Called JET, for the “JOBS 
Employment and Training Program,” the demonstration was established under the 
umbrella of the state’s PEACH (“Positive Employment and Community Help”) 
program, which also includes employment and training services under E&T and JOBS. 
The demonstration was established to further the following goals: 


e Facilitate the coordination of comprehensive training and skills 
development for public assistance recipients. 


e Provide necessary support services to public assistance recipients 
participating in training and making the transition to employment. 


e Assure that public assistance clients have real opportunities for job 
placement and career development. 


To further these goals, Georgia undertook several changes simultaneously. As 
described below, it implemented a statewide redesign of the E&T program to bring it 
closer to the JOBS model. Within the four demonstration counties, demonstration 
approval made it possible for Georgia to enhance the range and level of supportive 
services available to demonstration participants.! Doing so, it was believed, would 
enable participants to undertake the education and training they need to obtain 
employment and reduce dependency on Food Stamps. The demonstration also resulted 
in the provision of individual service planning and case management that was more 
intensive than was typical for E&T clients in non-demonstration counties. 


CONTEXT 

E&T Services Prior to the Demonstration 

The Food Stamp E&T Program is administered and operated by the Georgia 
Department of Human Resources (DHR), through the Division of Family and Children 


1To facilitate evaluation of the impact of enhanced supportive services, the state randomly 
assigned E&T participants in the demonstration counties to a treatment and a control group. Treatment 
group members were eligible for enhanced supportive services using the JOBS rules, while control group 
members followed JOBS rules for the assignment and sequencing of components but were not eligible for 
enhanced supportive services. 
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Services (DFCS). Prior to FY 1993, most mandatory work registrants were required 
to participate in two E&T components: group job search, followed by individual job 
search for a total 60-day required participation period.? As an option to the required 
Service sequence, individuals interested in education or training could enter an 
approved education or training activity after the completion of group job search. 
Optional service components included: adult education, work experience, vocational 
training, and Jobs Training Partnership Act (JTPA). For individuals entering optional 
components, case managers were required to complete employability assessments and 
develop individual employment plans. 


In March 1993, one month before the implementation of the demonstration, 
Georgia’s E&T program went through a substantial revamping statewide. This 
restructuring represented Georgia’s effort to conform its E&T and JOBS programs on a 
Statewide basis, to the extent allowed by federal regulations. Asa consequence of this 
effort, the E&T program moved from nearly total reliance on job search to the 
availability of a wide variety of activities to meet the needs of individual participants. 
The same service components are available under E&T and JOBS. These component 
activities include: 


e Assessment 

e Job readiness 

e Group job search 

e Individual job search 


e Education, including high school, basic and remedial education, GED, 
English proficiency, and post-secondary education 


¢ Vocational skills training, including job skills training, Job Corps, OJT 
from the JTPA system 


e Work experience. 
Under the revised statewide E&T program, no particular sequence of services is 


mandated. As in the old E&T program, an individual assessment and employability 
plan must be completed for any client who enters an education or vocational training 





2A more limited E&T program was operated in five “generic” counties that lacked separate E&T 
case managers. In the generic counties, income maintenance workers assigned mandatory work 
registrants to complete thirty days of individual job search. 


activity. Individual E&T counties have substantial discretion to draw on an array of 
local employment and training services in developing individual employability plans. 


After mandatory work registrants are called in for services, E&T case managers 
apply a set of state-designated individual exemption criteria, including: lack of 
transportation, lack of dependent care, temporary disability, homelessness, and good 
cause (a generic category for miscellaneous other reasons that might prevent program 
participation). Mandatory nonexempt work registrants are selected for the E&T 
program according to their priority target group status. Target groups (described 
below under the demonstration service design) are similar to the ones used in JOBS. 
Sanctions and supportive services are in accordance with E&T regulations, although all 
E&T participants are offered child care reimbursement at the same rate as in the JOBS 
program, with 100 percent-state funds used to supplement child care payments when 
they exceed the E&T maximum. 


Despite the state’s leadership in revamping the components of the regular E&T 
program and encouraging the use of a wider variety of services, group job search and 
individual job search continue to be the predominant service activities in practice, at 
least in the four comparison counties for which service utilization data were provided 
in the state’s process and implementation evaluation report. According to the 
information provided by the state, between July 1993 and June 1995, 78 percent of 
participants who had completed services in the four comparison counties had 
participated in group job search and 75 percent had participated in individual job 
search. In contrast, fewer than 5 percent of participants who had completed services 
had participated in education or training activities. These statistics may underestimate 
the actual rate of participation in education and training services among all program 
participants, because they were available only for individuals who had completed E&T 
services at the time the data were extracted. However, as described below, the 
demonstration counties exhibited a much higher rate of participation in education and 
training according to data extracted under the same conditions. 


Shifts in Non-Demonstration E&T Services During the 
Demonstration Period 


Since the startup of the demonstration in April 1993, little has been altered in the 
design or operation of the statewide E&T program. In one of the few changes, the 
number of E&T counties was reduced from 41 to 34, effective FY 1994, because of 
attrition among case manager staff and reductions in federal program resources. 
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Twenty-nine counties have full E&T programs, while the remaining five counties have 
“generic” job search services administered by income maintenance workers. Beyond 
these changes, individual E&T counties may have shifted their service designs, 


exercising the discretion they are provided under the state plan. 


DEMONSTRATION SERVICE DESIGN 

Initiation of the Demonstration Program 

Georgia was interested in applying for demonstration funds, because the 
demonstration was in line with its ongoing efforts to conform PEACH’s E&T and 
JOBS programs as closely as possible. It applied for and won authorization to operate 
the demonstration in 4 counties: Chatham, a large urban county, Clayton, a smaller 
urban county, Glynn, a county with medium density, and Gilmer, a rural county. 
Together, the demonstration counties contain about 16 percent of the state’s Food 
Stamp population. 


Implementation was planned for April 1993. In anticipation of this start date, 
E&T participants ceased being enrolled under the old program rules in these 4 counties 
as of December 1992. Staff training was held in March 1993 and referrals to JET 
began the following month, as planned. As part of its demonstration and evaluation 
plan, Georgia randomly assigned E&T participants in the 4 demonstration counties on 
the basis of the last digit of their social security number into either a treatment or 
control group. Those in the treatment group were eligible to receive the enhanced 
supportive services offered by the demonstration (described below), while those in the 
control group were limited to $25 per month for the reimbursement of transportation 
expenses. Participants in both the treatment and control groups were eligible for 
reimbursement of child care expenses at the JOBS rate. 


Targeting and Participation 


Consistent with the intent of the demonstration, mandatory work registration 
rules were changed to match those for JOBS participation. However, the state elected 
to continue to apply its E&T program criteria for individual exemptions, described 
above, as a second-stage screening process to mandatory work registrants selected 
using the JOBS work registration rules. This screening stage was similar to the 
identification of clients for whom participation would be non-feasible under JOBS 
rules. Application of the E&T exemption criteria made it possible for the state to 
eliminate exempt work registrants from the base used in computing its E&T 
participation rate standard. 


Also in keeping with JOBS, Georgia gave priority to enrolling and serving 
volunteers in the demonstration counties. In addition to giving priority to volunteers, 
the PEACH JET program, like the E&T program statewide, gave priority to: 


e Custodial parents under age 24 who have not completed high school or 
equivalent and are not enrolled in school, or who have less than 6 
months of work experience. 


e Primary wage earners in two-parent households. 


e Persons who have received Food Stamps for 36 of the previous 60 
months. 


e Self-initiated participants (those already enrolled in education or 
vocational training). 
e Registrants without a high school diploma or GED. 


e Registrants with less than six months work history. 


All four demonstration counties had sufficient service capacity to call in 
mandatory work registrants from all target groups. Several of the participating 
counties usually had sufficient capacity to call in all mandatory work registrants, as 
well. However, a high proportion of all individuals called in who reported for an 
initial screening were subsequently determined to be exempt or ineligible for services. 
Data supplied by Georgia in its process and implementation report show that 10,815 
persons were screened for services in the demonstration counties between April 1, 
1993 and March 31,1996. Of these, 1,535 (or 14 percent) had been enrolled. Of 
those not enrolled, most were found to be exempt due to: homelessness (23 percent); 
closure of their Food Stamps case (18 percent); disability (3 percent); transportation 
difficulties (1 percent); employment (7 percent); or for good cause or other reasons (10 
percent). Another 23 percent were closed as referred for sanction due to 
noncompliance. Seven percent were not enrolled for reasons that were not specified. 


In its process and implementation report, Georgia also reported the characteristics 
of persons who were enrolled in the demonstration and who had terminated from the 
program during the period from April 1993 to March 1996. Of these persons, 28 
percent were volunteers. Among both voluntary and mandatory participants, 81 
percent fell into one of the target groups noted above, including those who were: 


3These statistics included only enrollees who had terminated services at the time the data were 
extracted. 
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without a high school diploma or the equivalent (34 percent), individuals in self- 
initiated education or training (19 percent), long-term Food Stamp recipients (10 
percent), primary wage earners in two-parent households (7 percent), custodial parents 
under age 24 (7 percent), or with a work history lasting less than 6 months (4 percent). 


Elsewhere, results provided by the state evaluation report show that the majority 
of demonstration participants were females (74 percent), African-Americans (70 
percent), school dropouts (49 percent), and single (57 percent) or separated, widowed, 
or divorced (16 percent). It also became apparent to Georgia that substantial numbers 
of demonstration participants were persons who volunteered for the program primarily 
to obtain access to the child care assistance that the demonstration offered to 
participants in training or within the first 90 days of employment. 


Service Design 

As did other demonstration states, Georgia emphasized the provision of 
assessment, service planning, and one-on-one case management to demonstration 
participants. The service process begins with an group orientation session, at which 
the service offerings are described. Thereafter, assessment takes place, to review 
whether the individual qualifies for an individual exemption. For those who are not 
exempt or otherwise ineligible, the assessment covers the client’s education and 
employment history, aptitudes and interests, and potential barriers to achieving success. 


Based on assessment results, an individual employability plan is developed, 
which may involve one or several service activities, including job readiness training, 
job search, or education and other training. Data supplied by Georgia as part of its 
process and implementation report show that, of persons served in the JET counties 
who exited between April 1993 and March 1996, almost everyone was classified as 
having received an assessment (97 percent). Service components in which they 
participated included individual job search (39 percent), group job search (1 percent), 
job readiness training (2 percent), adult education (21 percent), ESL instruction (1 
percent), high school (4 percent), college (9 percent), vocational school (4 percent), 
work experience (3 percent), or other job skills training (3 percent). By contrast, as 
noted above, in the four comparison counties, assessment was much less likely to be 
provided (17 percent) and reliance on group or individual job search was much higher, 
with extremely low rates of participation in education and training components. 
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Some participants in the demonstration counties also were entitled to receive 
enhanced supportive services. As already described, JET participants were randomly 
assigned by the last digit of their Social Security Number to be in either the treatment 
or control group. Both groups received similar case management and service planning, 
but those in the treatment group received supportive services according to the JOBS 
supportive services plan, while those in the control group received the same supportive 
services to which E&T participants in non-demonstration counties are entitled. 


Enhanced supportive services consisted of a range of available supports, 
including reimbursements for transportation, meals, day care, tuition, and work-related 
expenses. For example, treatment group participants received up to $3 per day for 
transportation assistance, while others (i.e., control group members and those in non- 
demonstration counties) received at most $25 per month. Additionally, treatment 
group members could receive a meal allowance (up to $3 per day) and receive 
assistance for vehicle repair and insurance, medical care, dental services, eye care, and 
post-employment services, while none of these forms of support were available to 
others. Although all E&T and JOBS participants statewide received child care 
reimbursements at the same rate, treatment group members in the demonstration 
counties qualified for child care assistance during the first 90 days of employment, as 
part of post-employment services. An additional advantage enjoyed by treatment 
group members is that they could receive in-house ABE instruction developed and paid 
for by the demonstration, while control group members were referred to the local 
education agency, which usually offered instruction that was less intensive. 


Evolution of the Demonstration Design Over Time 

Between April 1993 and October 1994, the JET service design remained very 
stable. Over time, however, word apparently spread among Food Stamp beneficiaries 
that the E&T program offered generous child care benefits, and, as a consequence, an 
increasing number of individuals volunteered for services who were interested solely in 
receiving child care assistance. Additional changes in the relative frequency of 
utilization of job search versus education and training services may also have occurred 
during FY 1996, as part of the Georgia “Work First” initiative for its welfare-to-work 


4State administrative staff do not consider this assistance “transitional child care services,” which 
is not an allowable expense under the conformance demonstration. Rather, because the state keeps the 
E&T case open during the first 90 days after employment, child care assistance after employment is 
considered part of the planned post-employment services available to the active caseload. 


A-7 


programs. However, these changes were not described in the state’s process and 
implementation report. 


PROGRAM COSTS 

Costs associated with serving demonstration participants include personnel, 
administrative, and overhead expenses associated with program operations, which were 
funded from the regular E&T budget, and expenditures for supportive services, which 
were funded through a combination of regular E&T and cooperative agreement funds 
associated with the demonstration. Although not provided to SPR, Georgia’s regular 
personnel, administrative, and overhead costs associated with the demonstration, which 
are funded under a departmental cost allocation plan, are judged to be comparable for 
demonstration and non-demonstration counties of equivalent size. 


Regular E&T funds were also used to provide all supportive services to control 
group members in the demonstration counties, as well as for all services in non- 
demonstration counties. During State Fiscal Years 1994 and 1995, regular E&T 
expenditures for supportive services to control group members in the demonstration 
counties totaled $265,442, most of which resulted from child care assistance payments. 
In contrast, supportive service costs for all E&T participants in the four comparison 
counties totaled only $99,785 during the same period. Thus, the increased use of 
education and training components within the demonstration counties even among 
control group members (associated with the provision of individual case management 
services to all participants and the inclusion of voluntary participants) resulted in 
significantly higher supportive service expenditures than in the non-demonstration 
counties. 


Cooperative agreement funds were used to provide all supportive services to 
members of the treatment group in the demonstration counties. During State Fiscal 
Years 1994 and 1995, supportive services expenditures for experimental group 
members totaled $406,543. Of this amount, 14 percent was used for transportation 
assistance, 10 percent for meals, 62 percent for child care assistance, 3 percent was 
used for incidentals, and 11 percent for the purchase of on-site remedial adult education 
services for treatment group members. When compared with budgeted amounts (i.e., 
planned expenditures), the expenditure figures show that Georgia spent substantially 
more than the anticipated amount for child care expenses, and substantially less than 
the anticipated amount for every other form of supportive service. 


Cooperative agreement funds are also being used to support the evaluation of the 
demonstration. Although most of the evaluation activities have been carried out by the 
staff of the Georgia Department of Human Resources, $46,680 of cooperative 
agreement funds are planned to cover the costs of the preparation of the state’s cost and 
impact evaluation report by an independent consultant. 


SUMMARY OF KEY DEMONSTRATION FEATURES 

In contrast to its regular E&T program, Georgia’s JET demonstration 
emphasizes: (1) participant access to a more flexible sequence of component services, 
in combination with individualized assessment and one-on-one case management for all 
clients, and (2) more generous supportive services, including enhanced assistance for 
training-related expenses and post-employment services. In combination, these 
demonstration features resulted in (3) more frequent utilization of education and 
training activities by participants. However, because of the priority placed on serving 
volunteers in self-initiated training combined with the generous supportive services 
available to all participants, Georgia found that JET was frequently accessed by 
participants who were primarily interested in child care assistance. 
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THE HAWAII PRIDE DEMONSTRATION 


GOALS AND OBJECTIVES 

Hawaii’s Food Stamp Employment and Training/JOBS Conformance 
Demonstration, in operation on the island of Oahu between November 1993 and 
September 1996, is called PRIDE (“Positive Response in Developing Employment”). 
The Hawaii Department of Human Services (DHS) described overall PRIDE goals as 
“maximizing the employability of Food Stamp recipients and reducing Food Stamp 
dependency through improved consistency and coordination between the JOBS and 
E&T programs.” Additional demonstration objectives included: 


e Reducing administrative and service costs due to coordination across 
programs. 


e Enhancing E&T services through the use of a new case management 
system, the addition of new barrier removal and family social support 
services, and an expanded emphasis on educational activities. 


e Decreasing program errors due to reduced complexity of and conflicts 

between program regulations for the E&T and JOBS programs. 

Although the PRIDE program did not call for the consolidation of PRIDE and 
JOBS services, the demonstration was intended to increase overall fairness by offering 
all public assistance recipients “the same realistic and meaningful opportunities to 
achieve self-sufficiency.” The key features of PRIDE’s service strategy included the 
introduction of a comprehensive case-management based approach designed to link 
families and individuals to needed support services for the removal of psycho-social 
barriers to employment, followed by the provision of needed employment preparation 
training, basic education, and vocational training services. As stated by DHS in its 
process and implementation report, “the PRIDE program is attempting to demonstrate 
that a JOBS-like E&T program will prove to be more beneficial to the participants due 
to its comprehensive and multi-disciplinary approach.” (p. 17) 


CONTEXT 

E&T Services Prior to the Demonstration 

Prior to the demonstration, DHS contracted with the Hawaii Department of Labor 
and Industrial Relations (DLIR) to operate E&T services through designated E&T staff 
housed in its local Employment Services offices on Oahu, Kauai, Hawaii, and Maui. 
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All mandatory work registrants were referred by DHS eligibility workers to the E&T 
Program. After an initial assessment interview, DLIR E&T staff developed an 
employability plan and directly provided or referred participants to one or several 
appropriate components. E&T services available on Oahu prior to the demonstration 
included: 


e Individual job search. Up to two 8-week job search periods could be 
required in any 12-month period. 


e Job search skills training. At least 24 hours of job search training over 
a 6-day period was planned. 


e Non-reimbursable referral to basic education services on a space 
available basis. 


e Non-reimbursable referral to vocational skills training on a space- 
available basis. 


e Work experience. Up to two 13-week work assignments with a public 
or non-profit agency could be required. 


Individual job search was the most frequently used service on Oahu prior to the 
demonstration and continues to be the most frequently used service in the non- 
demonstration counties. As reported in Hawaii’s process and implementation report (p. 
29), 89 percent of enrollees in the regular E&T program on the island of Hawaii 
between November 1993 and June 1995 participated in the job search component in a 
typical month, 15 percent participated in job search skills training, and only 24 percent 
of enrollees participated in all other components combined. 


Shifts in Non-Demonstration E&T Services During the 
Demonstration Period 


Between November 1993 and June 1995, the E&T program in Hawaii, Maui, and 
Kauai operated without major organizational or service redesigns, using the pre- 
demonstration service design described above. E&T operations were disrupted in July 
1995 as a result of reductions in state funding, and new service delivery arrangements 
had to be negotiated with DLIR and/or other providers. The Hawaii process and 
implementation report did not provide detailed information about the resulting changes 
in E&T program design. 
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DEMONSTRATION SERVICE DESIGN 

Initiation of the PRIDE Demonstration 

The PRIDE demonstration was implemented throughout Oahu in November 
1993, after a brief transition period during which the old E&T program was suspended 
and the E&T caseload was terminated. At the time the demonstration was initiated, 
Oahu accounted for slightly over 70 percent of all Food Stamp cases in the state. 
Initially, the state planned to expand the demonstration to include the entire state after 
the first year of the demonstration. However, due to subsequent state budget cuts, 
PRIDE was not expanded beyond Oahu. 


Targeting and Participation Patterns 

Like other demonstration states, Hawaii implemented the JOBS work registration 
rules for demonstration participants. These rules require participation by persons 
caring for children over the age of three; custodial parents under age 20, regardless of 
the age of the child; unemployment compensation recipients and applicants, and 
students in self-initiated education or training. As a result, the number of individuals 
in the work registrant pool during the PRIDE demonstration was expected to increase 
and the characteristics of work registrants were expected to differ from the pre- 


demonstration period. 


For the PRIDE demonstration, Hawaii tested a targeting model that was 
dramatically different from the previous E&T approach. Rather than calling in all 
mandatory work registrants for service, the PRIDE program called for selective 
targeting of “hard-to-serve” individuals who meet criteria parallel to the JOBS 
targeting criteria, as well as several additional priority criteria. Priority groups 
targeted for participation included work registrants who: 

e Had worked less than three consecutive months in the six months prior 

to work registration. 


e Had received Food Stamps for at least 12 of the 24 months prior to 
work registration. 


e Were homeless. 


e Were between the ages of 18 and 24 and lack a high school diploma or 
the equivalent. 


e Had a primary language other than English. 


The PRIDE design called for 55 percent of work registrants referred to PRIDE to 
belong to at least one of these target groups, and the remaining 45 percent to be 
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referred randomly from the total work registrant pool. The program called for priority 
to be given to individuals who volunteer for services prior to call-in. 


Hawaii’s Implementation and Process Report provides descriptions of individuals 
who participated in PRIDE over the twenty month period from November 1993 
through June 1995. During this period, there were a total of 12,721 mandatory work 
registrants in Oahu. During the same period, a total of 1,260 individuals (9 percent of 
the work registrant pool) participated in PRIDE. Of these participants, 723 (57 
percent) were categorized as “mandatory volunteers” because they volunteered before 
being called in; 54 (4 percent) were volunteers exempt from work registration, and 483 
(38 percent) were mandatory work registrants who participated after being called in for 


services. 


Of the 1,260 PRIDE participants during this 20 month period, 72.8 percent were 
individuals from households without dependent children and 52.5 percent were male. 
Individuals from target groups comprised 71 percent of PRIDE participants, with 
representation from the following target groups as follows!: 


e Worked less than three months out of the previous six months (33 
percent of all PRIDE participants). 


e Received Food Stamps in 12 of the past 24 months (30 percent). 

e Homeless (12 percent). 

e Age 18 through 24 without high school completion (6 percent). 

e Primary language other than English (5 percent). 

Because of limitations in data availability, Hawaii’s process and implementation 
report does not reveal how the demonstration affected the overall volume or 
characteristics of work registrants or E&T participants on Oahu, nor does it provide 
information about how the characteristics of demonstration participants differed from 
the characteristics of E&T participants in the comparison county (Hawaii). More 
information on how the demonstration affected participation patterns should be 
available from the state’s impact evaluation report. 


1Target group membership counts are duplicated counts, as an individual can fall in more than one 
category. 
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Service Design 


One of the stated objectives of the PRIDE demonstration was to offer more 
intensive services to Food Stamp work registrants. The first step in the PRIDE 
demonstration, modeled after the JOBS program, was to provide a detailed assessment 
of participants’ health, psycho-social, financial, legal, family, and education- or skills- 
related barriers to employment, followed by referral to “barrier removal” services 
provided by community agencies specializing in various issues or problems. To 
implement this sequence of services, each PRIDE unit within DHS was staffed with a 
trained social worker in addition to DHS case managers, and had access to a public 
health nurse for referral of participants with identified health problems. 


Hawaii’s process and implementation report indicates that 301 individuals 
completed an assessment to determine barriers to employment in the first 12 months of 
PRIDE operation. Identified barriers, in order of frequency, included lack of 
transportation, lack of formal training, lack of job-finding skills, job of specific job 
skills, health problems, low self-esteem, poor coping skills, problems with 
interpersonal relationships and lack of support systems. (p. 42) However, during the 
demonstration, DHS found that the number of PRIDE participants who voluntarily 
participated in barrier removal services after assessment through referral to outside 
agencies was relatively low. It was difficult for project staff to assess whether this was 
because PRIDE participants did not feel that their barriers were significant, or whether 
they had a low interest in participating in services to address identified barriers. 


Once individuals had completed the assessment and removal of barriers and were 
determined by their DHS case manager to be ready for employment-related services, 
PRIDE participants were referred to DLIR “employment counselors.” To address the 
limited self-esteem and low self-confidence of many PRIDE participants, the program 
contracted with the University of Hawaii Department of Counselor Education for a six- 
hour orientation session that emphasized self-esteem building and empowerment. 
Rather than automatically placing all participants in individual job search, PRIDE 
employment counselors worked with participants to develop individual career plans and 
placed participants in component services most relevant to their career plans. Like 
JOBS participants, all PRIDE participants were required to enroll in component 
activities for a minimum of 24 hours per week. 


Pre-employment and job search services available to meet the needs of PRIDE 
participants included: 
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A 6-week pre-employment preparation curriculum provided by an 
outside contractor—the City and County of Honolulu, Work Hawaii 
Program—that focused on esteem building, anger management, goal 
setting, self-awareness, skills exploration, and attitude adjustment. 


A 2-week job search skills training class provided by the Employment 
Training Center affiliated with University of Hawaii’s community 
college system. 


Individual job search was still available for participants who had 
completed education or training and were ready to seek employment. 


Additional services, available through referral to community resources, included: 


Because the PRIDE program did not have as much funding as the JOBS program 


Educational activities, including ESL classes, remedial education, and 
adult basic education. 


Post-secondary education and training, including referral to community 
vocational programs, two-year certificate, and four-year degree 
programs. 


Work experience. 


: 


it was not able to offer PRIDE participants access to the more intensive adult basic 


education programs (24 hours per week) implemented for JOBS participants. Instead 


PRIDE participants were referred to community-based programs in which instruction 


was usually available 5 to 7 hours per week. 


As reported in Hawaii’s process and implementation report (p. 29), during a 
typical month between November 1993 and June 1995, 387 individuals participated in 


PRIDE. 


In an average month, 


16 percent of the participants were processed for program intake by 
DHS 


9 percent received an assessment of psycho-social barriers by the DHS 
social worker 


24 percent were assessed for employment readiness by their DHS case 
manager prior to referral to a DLIR employment counselor 


12 percent participated in the 6-hour motivational orientation 


6 percent participated in the 6-week pre-employment preparation 
curriculum 


3 percent participated in a job search activity 


5 percent participated in an education activity 
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e 10 percent participated in a post-secondary education or skill training 
activity, including 2 percent who participated in self-initiated training 


e 3 percent participated in work experience.” 


Hawaii’s impact evaluation should be able to provide more detailed information 
about the patterns of service utilization among PRIDE participants, compared to E&T 
participants during the pre-demonstration period and in the comparison county. 


Supportive services availability was also expanded as a result of the PRIDE 
demonstration to conform to the JOBS supportive service design (excluding transitional 
child care). As a result, child care expense reimbursements available to PRIDE 
participants increased substantially over the levels available under the E&T program, as 
did the monthly amounts available for reimbursement of transportation expenses. 
Expanded supportive services available to PRIDE participants also included small one- 
time payments for training-related expenses, such as work tools or clothing, and 
payment of a variety of one-time work-related expenses upon exit from the program. 


Evolution of Demonstration Design Over Time 

Between November 1993 and March 1995, the PRIDE program operated without 
major design changes. However, during the PRIDE demonstration period, the state 
JOBS program underwent a full-fledged review and redesign. During 1995, in 
response to growing public concern about welfare programs supporting individuals 
through four years of training, DHS implemented several key changes to the JOBS 
program design, which were incorporated into the design of the PRIDE demonstration 
as well. These changes included: 


e A new requirement that all PRIDE participants work at least 8 hours 
and preferably 16 hours a week in private sector unsubsidized 
employment. This requirement was imposed on all participants, 
including those in education and training components, beginning in 
April 1995. 


e A revision to tighten the conciliation procedures used by the PRIDE 
program to require participants cited for non-cooperation to demonstrate 
compliance during a probation period, beginning in April 1995. 


2The percentages provided by Hawaii in the process and implementation report do not add to 100 
percent. This suggests that, in a typical month, approximately 15 percent of the PRIDE enrollees were 
not actively participating in a component. Perhaps they were waiting for a component to start, or had 
recently completed an activity. 


e A redesign of all PRIDE program features and service delivery 
arrangements in response to a dramatic reduction in state funding, for 
both PRIDE and JOBS beginning in July 1995. The program service 
emphasis shifted from barrier removal and employability development 
to immediate job development. Beginning August 15, 1995, all 4-year 
training plans were phased out or converted to two-year training plans. 
As part of the changes that took place in July 1995, DHS terminated its 
contracts with most of its outside service providers, including DLIR, 
and undertook in-house operation of the PRIDE program. 


PROGRAM COSTS 

Statewide expenditures on the E&T program are summarized below. The state 
did not provide separate information on the cost of operations in the demonstration 
county of Oahu. During FFY 1994 and FFY 1995, Hawaii substantially increased the 
level of state and federal expenditures to support the increased costs of the intensified 
case management and barrier assessment/removal services offered to PRIDE 
participants. Most of the increased costs of the demonstration occurred in the costs of 
program operations. Only modest increases were experienced in dependent care costs, 
while reimbursements for transportation and other expenditures actually decreased. 
State budget cutbacks necessitated the reduction of E&T expenditures planned for 
FFY 1996, requiring the elimination of the contract with the Department of Labor and 
Industrial Relations, as described above, and the streamlining of program services. 


Demonstration funds were used to support the costs of dependent care and other 
supportive services in the demonstration county, pay for the delivery of PRIDE 
orientation sessions emphasizing self-esteem and motivational skills by an outside 
contractor, pay for pre-employment preparation sessions also provided by an outside 
contractor, and purchase individual education and training services for demonstration 
participants. In addition, demonstration funds were used to support the costs of the 
independent evaluation. 





3Because of the dramatic changes in both PRIDE and E&T design and operations after July 1995, 
a decision was made to include only the 20-month period from November 1993 through June 1995 in the 
state’s evaluation of the PRIDE demonstration. 
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Year 


FFY 1993 | $136,192 $ 679,899 $ 1,890 |$ 71,410 | $ 889,391 
FFY 1994 | $205,812 $1,164,666 $ 5,913 |$ 32,620 | $1,409,011 
FFY 1995 | $191,016 $1,618,103 | $154,168 $ 3,686 |$ 24,200 | $1,991,173 


FFY 1996 | $204,450 $1,054,965 | $300,000 $ 13,612 |$ 37,260 | $1,610287 


planned 
SUMMARY OF KEY DEMONSTRATION FEATURES 











Among the key program features tested by the PRIDE demonstration are the 
following: (1) an effort to identify a wide range of employment barriers faced by 
participants and link individuals to community agencies that can help address those 
barriers; (2) the design and delivery of preemployment services to address their 
particular motivational and self-esteem issues as well as to provide participants with 
improved job seeking skills; (3) an emphasis on participation by work registrants from 
priority target groups that can benefit from more intensive E&T services and by both 
mandatory and exempt work registrants who volunteered for services; and (4) an 
emphasis on encouraging participation in education and training services, particularly 
for participants who have not completed high school. 


During the demonstration period, the rest of the state’s E&T program continued 
to operate using the program design and targeting policies typical of the pre- 
demonstration period, with required participation by all mandatory work registrants in 
a program that emphasized an independent job search component. 
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THE MISSOURI JET DEMONSTRATION 


GOALS AND OBJECTIVES 

The JET Program (“JOBS-Employment and Training Demonstration”) is 
Missouri’s Food Stamp Employment and Training/JOBS conformance demonstration. 
Originally intended to operate in 20 counties, implementation difficulties compelled 
Missouri to scale back the JET program to an 8-county area. Goals of the 
demonstration project were varied and included: 


e Realizing increased efficiency and reduced duplication of effort in 
program administration and operations. 


e Strengthening coordination with JTPA and other state and local 
education and training providers to increase the range and intensity of 
service options available to participants. 


e Improving the match between E&T services and participant interests 
and needs through a thorough client assessment followed by individual 
employability planning, necessary support services, and ongoing case 
management. 


e Encouraging voluntary participation in relatively high intensity and 
long-term activities. 


e Increasing targeting to the least job-ready portion of the Food Stamp 
work registrant population and those most likely to be long-term 
recipients. 


In carrying out these goals, Missouri moved decisively towards intensive case 
management. A careful assessment process and development of a service plan matched 
to each participant’s needs and interests were key components of the case manager’s 
role, as was shepherding clients through the training process. As a consequence of the 
resource-intensive efforts expended on each individual, relatively few participants have 
been served under the demonstration to date. 


CONTEXT 

E&T Services Prior to the Demonstration 

Prior to the demonstration, the Division of Family Services (DFS) in the 
Department of Social Services (DSS) operated the Missouri Employment and Training 
Program (METP) through two contracts. The first and largest contract was with the 
Division of Employment Security (DES) of the Department of Labor and Industrial 
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Relations for the provision of job search services to mandatory work registrants 
referred by DFS eligibility case workers. E&T case workers within DES were 
provided with monthly lists of Food Stamp work registrants by DFS. Mandatory work 
registrants who responded to a summons letter issued by DES were provided with an 
initial assessment followed by participation in one or more E&T components. Job 
search training was the most frequently used component. For most METP 
participants, job search training consisted of a regimen of self-directed job search that 


could last up to 8 weeks. 


Additional METP components included basic education, job skills training, on- 
the-job training, work experience, and job development and training. At the time of 
application or recertification, eligibility workers could refer interested work registrants 
to the local JTPA system. In addition, for mandatory work registrants called in by 
DES, additional component services could be provided through non-financial referral 
linkages with other programs. Where the DES case worker thought it advisable, 
participants could be referred to the local JTPA program for regular JTPA services, or 
to local education agencies funded by the Department of Elementary and Secondary 
Education for basic skills remediation. A non-financial referral agreement with the 
Department of Elementary and Secondary Education called for local education agencies 
to develop individual education plans for METP participants referred to ABE/GED 
training. However, individuals receiving services through non-financial referrals were 
not tracked as part of the METP-DES system. 


A second DFS contract for the delivery of services to METP participants was 
with the Division of Job Development and Training (DJDT) of the Department of 
Labor and Industrial Relations, which oversees JTPA at the state level. Prior to 
FY 1993, the METP-DJDT funds were allocated among five local JTPA service 
delivery areas to provide JTPA-related services to METP work registrants. The 
METP-DJDT contract was operated as a separate program from the METP-DES 
service delivery system. JTPA administrative entities were responsible for recruiting 
and enrolling their own Food Stamp/non public assistance participants, who were then 
reported to DFS as METP participants. 


Shifts in METP Services During the Demonstration Period 


METP was substantially redesigned at about the time the demonstration began. 
Indeed, the process of competing for demonstration funds caused Missouri to 
reconsider and modify its regular E&T program in important ways. These changes 


C-2 


included: (1) the redesign of the METP service components statewide to conform to 
the state JOBS program; (2) an expansion of the services provided by DES to include 
more extensive training in job readiness skills; and (3) the redesign of the METP- 
DJDT funding to include all JTPA service delivery areas in the state and emphasize a 
wider range of JTPA-funded services. 


After FY 1994, DFS redesigned the service components available to all METP 
participants in the non-demonstration counties to conform to the state’s JOBS 
components. Available components include: job clubs/job readiness training, job 
search assistance - independent job search, adult basic education, school-to-work 
transition, post-secondary education, job skill training, work experience, and on-the- 
Job training. As a result of these changes, the services components available to METP 
non-demonstration participants and participants in the JET counties were very similar, 
with the exception of the provision of intensive case management and availability of 
enhanced transportation assistance and other work-related supportive services for JET 
participants, as described below. 


Prior to the onset of JET, DES ran most METP-DES participants through 8 
weeks of self-directed job search as their sole E&T activity. Participant progress in 
self-directed job search was monitored at the end of the fourth and eighth week of 
participation. Starting in FY 1994, however, DES redesigned its METP services to 
require all participants to first attend 3 half-day sessions of training in job readiness and 
job seeking skills (referred to as “job club”). Once the job club was concluded, 
independent job search commenced. During independent job search, participants are 
required to make a minimum of 12 employer contacts during each of two 4-week 
periods of job search and meet with E&T staff at the end of each period to discuss their 
progress. 


Another important change implemented at about the time of the onset of JET was 
a redesign of the JTPA services provided under the METP-DJDT contract. First, the 
DJDT Division decided to allocate this funding to all SDAs statewide, rather than 
reserving the funds for a subset of all SDAs. Second, DFS encouraged the 
participation of METP participants in the full range of JTPA services under this 
contract. (There is some evidence that METP-funded services from the JTPA system 
prior to this change had emphasized job development and placement services, rather 
than a broad range of vocational training services.) 
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Numbers provided by DFS as part of its process and implementation evaluation 
reports show that JTPA is indeed providing a wide range of services to METP 
participants in non-demonstration counties. During the period from October 1, 1994 to 
September 30, 1995, DES served 6,288 METP participants, through its sequence of 
job clubs and job search. In comparison, JTPA served nearly as many METP 
participants (5,024 participants). Vocational skills training was by far the most 
common training service received by METP participants through the METP-DJDT 
contract, although substantial numbers received on-the-job training, paid work 
experience, or other services or combinations of services. No figures are available on 
the number of non-demonstration METP participants served through the Department of 
Elementary and Secondary Education. 


DEMONSTRATION SERVICE DESIGN 

Initiation of the Demonstration Program 

JET was ushered in after a somewhat rocky start. As part of its intended design, 
Missouri’s DFS planned to hire in-house JET case managers who would be responsible 
for client recruitment, service planning, and case management in the planned 20-county 
demonstration area. This was initially expected to lead to full statewide consolidation 
of the E&T and JOBS delivery systems. However, the DFS job slots were not 
approved by the Missouri legislature during its 1993 session, forcing DSS to consider 
alternative arrangements. 


To initiate demonstration services, a contract was signed with the state DJ Di 
which in turn subcontracted with Private Industry Councils (PICs) in 2 service delivery 
areas—the St. Charles Office of Employment and Training, which was to operate the 
demonstration in St. Charles County, and the SEMO Private Industry Council, which 
was to operate in 7 counties in Missouri’s Bootheel.! Because of the delay caused by 
the failure of the state legislature to authorize the sought-after positions, the 
demonstration was quite late in becoming operational—the first 3 case managers were 
not hired and trained until February 1994 (1 was hired by St. Charles and 2 by SEMO) 
and the first client was not enrolled until March 1994. As an additional consequence of 





‘As described later in this narrative, DFS subsequently began contracting directly with the Private 
Industry Councils responsible for the local JTPA services delivery areas for JET services. This resulted 
in a significant reduction in administrative costs. 


C-4 


the delay, Missouri found it impractical to expand the operation of the demonstration 
beyond the initial 8-county area. 


Targeting and Participation 

In keeping with the spirit of the demonstration, Missouri applies the JOBS work 
registration rules to demonstration participants. In other demonstration states, the 
application of these rules was expected to lead to increases in the size of the work 
registrant pool. Curiously, however, no such increases were in evidence in Missouri. 
In the year prior to the onset of the demonstration, approximately 12,300 persons were 
classified as mandatory work registrants in the 8-county demonstration area. In the 
first year after the demonstration was implemented, however, only about 10,500 
persons were classified as mandatory work registrants in the same 8 counties, perhaps 
as a result of improvements in the local economy. 


Among persons who are classified as mandatory work registrants, Missouri’s JET 
program gives priority to persons who are deemed “hardest to employ” or “not job 
ready.” These concepts represent an elaboration and extension of the JOBS priority 
target groups and include: 


e Former participants in Missouri’s JOBS program (called FUTURES) 
who had lost their AFDC eligibility. 


e Those who lack a high school degree or its equivalent, have little or no 
work experience, and/or received Food Stamps for 24 out of the last 36 
months. 


e Displaced homemakers or former AFDC recipients. 


e Absent parents with child support claims pending with DFS. 


Work registrants are classified by the DFS caseworker according to this priority 
system at the time the individual applies for Food Stamps. The target group 
identification of each individual can thereby be included in the monthly computer 
listing sent to the JET case managers, from which selections for call-in are made. The 
first of the target groups identified above constitutes the highest priority, and these 
individuals are automatically selected for JET. Beyond that, JET places a high 
emphasis on serving volunteers, especially those who are also in one of the above 
target groups. Volunteers can be identified because the monthly computer listings 
prepared by DFS will include a special notation for individuals who expressed an 
interest to the DFS caseworker in receiving employment and training services. Beyond 
this, the JET case managers recruited applicants by posting flyers in locations likely to 
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be frequented by Food Stamp recipients (e.g., in DFS and ES offices) and giving 
speeches at periodic public forums. If vacancies for the JET program still exist after 
former FUTURES participants and volunteers have been enrolled, case managers call 
in work registrants from among non-volunteers according to the priority system 
identified above. 


Data supplied by Missouri as part of its process report show that just over one- 
half of JET participants served during the Federal Fiscal Year 1995 (FFY 95) were 
either mandatory work registrants or volunteers belonging to one of the identified 
target groups. Results also show that, in keeping with the JET program’s selection 
criteria, volunteers made up a high proportion of the caseload. Of the 297 persons who 
were served in JET in FFY 95, 180 (or about 61 percent) were classified as either 
mandatory or exempt volunteers. By contrast, almost no volunteers were served in the 
non-demonstration counties during the same period. 


Another dramatic difference between demonstration and non-demonstration 
counties was in the rate at which Food Stamp recipients were served. During FFY 95, 
just 123 mandatory work registrants were served in JET, or about 1.2 percent of the 
total pool of 10,168 mandatory work registrant in the demonstration counties; the 174 
exempt volunteers made up the remainder of the total JET caseload. By contrast, 
11,276 of 54,383 mandatory work registrants were served through METP in the non- 
demonstration counties, representing a service rate of about 20.7 percent; additionally, 
several dozen exempt volunteers were also served. 


Perhaps because of these differences in eligibility, targeting, and recruitment 
rules, the characteristics of program participants also varies somewhat between 
demonstration and non-demonstration counties. Although Missouri cannot provide 
complete information about METP participants in non-demonstration counties, it 
concludes that JET participants are much more likely to be female and white than their 
non-demonstration counterparts. 


Service Design 


A hallmark of the demonstration design has been intensive case management and 
service planning. This process begins with an orientation session to which prospective 
JET participants are invited. Generally, those who volunteer after attending the 
orientation session are enrolled in JET. Participants are given a comprehensive 
assessment of their employment and training and other needs. This assessment 


typically entails an in-depth interview with the case manager and testing for basic skills 
and vocational interests and aptitudes. Based on these assessment results, an individual 
employability plan (IEP) is developed jointly by the participant and case manager. 


In conformance with the IEP, the participant may thereafter be assigned to: 


e Classroom training, including enrollment in a GED or ABE program, 
English as a Second Language, a vocational school or college for post- 
secondary education or training, or a curriculum called “Parents as 
Teachers.” 


e Work experience, consisting of non-paid employment from which the 
participant is to gain work experience and job skills. 


e On-the-job training, consisting of employment partly paid by the 
program and with the expectation that permanent employment will 
ensue. 


e Job development/job placement, including help finding job vacancies 
and instruction in job search, interviewing techniques, and preparing a 
resume, and participation in peer support groups. 


Different types of the activities described above can be undertaken by the same 
participant, either sequentially or concurrently, depending on their needs. However, 
participation for 20 or more hours per week in one or more components is generally 
required. In principle, most of these service options were also available to E&T 
participants in non-demonstration counties, at least since the revamping of METP 
coincident with the implementation of JET a few years ago (see Context, above).? In 
practice, however, METP in the non-demonstration counties tends to rely more heavily 
on job readiness and job search training as a stand-alone activity, while JET more often 
enrolls individuals in education or occupational skills training. For example, 27 
percent of JET participants received GED or other basic skills training, 23 percent 
received post-secondary education, 8 percent received work experience, and 21 percent 
received other job skills training. Comparable figures for METP participants showed 
that 63 percent received only job readiness or job search training, while 34 percent 


received job skills training. 


2Service components that are available to JET participants but not to METP participants in the 
non-demonstration counties include: English as a Second Language, Parents as Teachers, and Peer 
Support. 


Another important element of the JET service design was the intensity of the case 
management that was provided. The three JET case managers each had maximum 
caseloads of approximately 40 participants. Their duties included: conducting 
orientation and assessment, developing the IEP in partnership with the participant, 
referring the participant to services and other activities, and providing periodic follow- 
up while the participant was enrolled and assistance for up to 90 days after eligibility 
expired. Generally, participants continue to be enrolled in JET and work with their 
case managers until either the goals of their IEP are met and employment results, or the 
person voluntarily chooses to exit or loses eligibility for Food Stamps. Missouri’s 
process report describes JET’s intensive and ongoing case management as “the single 
greatest difference from the regular METP program.” 


Under the demonstration, JET participants also were entitled to receive a more 
generous allotment of supportive services. Transportation assistance could amount to 
as much as $70 per week (compared to $25 per month under METP), and 
reimbursement could also be obtained for work-related expenses, including uniforms, 
tools and equipment, and necessary auto repairs, up to a limit of $350 per 12 months 
(or more, if special approval was obtained). However, child care assistance was 
generally not available to JET participants, until quite recently. 


Another important distinguishing characteristic of Missouri’s demonstration was 
that, as of April 1996, sanctions had never been applied for noncompliance with JET 
participation requirements. This fact, which represents a rather glaring departure from 
conformance with JOBS, appears to have come about de facto, rather than as a matter 
of explicit policy, because case managers felt that the use of sanctions would have a 
“chilling effect” on a program that emphasized participation by volunteers and would 
have served to discourage potential participants from volunteering. 


Evolution of the Demonstration Design Over Time 


Once the demonstration become operational and case managers built up their 
caseloads, JET operated without major change through FFY 95. Subsequently, some 
relatively minor changes were introduced. First, DFS began contracting for the 
services of the JET case managers directly with the two PICs involved, rather than via 
the intermediary of the state Division of Job Training and Development. Contracting 
with DJDT made sense when it appeared as if the demonstration would be operating in 
20 counties (in 8 different SDAs). However, when it became clear that the scope of 
the demonstration would entail only 8 counties in just 2 SDAs, DFS realized that 
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dealing directly with the PICs involved could eliminate a layer of administrative 
expense. 


Second, JET case workers in St. Charles County came to the realization that the 
lack of child care assistance severely curtailed access to JET for single parents with 
young children. Accordingly, it pressed to be allowed to provide child care assistance 
as a supportive service and recently was granted permission to do so. By contrast, the 
JET staff in the 7 counties operating the demonstration as part of SEMO still feel that 
the lack of child care assistance is not hampering demonstration operations, perhaps 
because extended families in its primarily rural service area make the need for child 


care assistance less pressing. 


Finally, very recently DFS realized that sanctions were not being applied for non- 
conformance within the demonstration project area and instructed case managers that 
sanctions should be applied when warranted, consistent with JOBS rules. 


PROGRAM CostTs 

JET is supported wholly through Missouri’s $500,000 in Cooperative Agreement 
Funds. These funds have been used to pay for: state and local administrative and office 
support, case management staff, employment and training services purchased as part of 
individual employability plans, and supportive services provided to JET participants. 
No demonstration funds are being used to support the evaluation, which is being 
performed in-house by DSS. 


Because of its late start-up, Missouri was quite slow in recording its first project 
expenditures. However, since JET became operational in October of 1994, funds have 
been spent according to the following schedule: 


Period Expenditures 
10/94 to 12/94 $70,127 
1/95 to 3/95 112,590 
4/95 to 6/95 37,841 
7/95 to 9/95 27,678 
10/95 to 12/95 51.411 


TOTAL (as of 12/95) $299,647 
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For FFY 95, funds were used to provide services to 297 participants, or approximately 
$750 on a per participant basis. In contrast, Missouri’s regular E&T expenditures in 
FY 95 amounted to just over $1 million, used to serve just over 11,312 METP 
participants, or about $100 per person. 


SUMMARY OF KEY DEMONSTRATION FEATURES 

Key elements of the JET service design include: (1) a strong emphasis on serving 
volunteers, (2) an emphasis on comprehensive assessment, careful service planning, 
and ongoing case management for each participant, and (3) access to a wider range of 
employment and training services than was typical for METP participants prior to the 
demonstration or is typical in non-demonstration counties. As a consequence of these 
attributes, JET served relatively few participants and a much smaller proportion of the 
demonstration area’s mandatory work registrants than was customary in the non- 
demonstration counties. 
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APPENDIX D: 

THE SOUTH DAKOTA FAMILY 
INDEPENDENCE FOOD STAMP 
E&T/JOBS CONFORMANCE 
DEMONSTRATION 
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THE SOUTH DAKOTA FAMILY INDEPENDENCE 
FOOD STAMP E&T/JOBS CONFORMANCE 
DEMONSTRATION 


GOALS AND OBJECTIVES 

South Dakota’s Family Independence Food Stamp Employment and 
Training/JOBS Conformance Demonstration was implemented statewide in all 20 
counties in which E&T services are offered. As stated by the South Dakota 
Department of Social Services (DSS), the objectives of the demonstration were to 
improve the operations, effectiveness, and efficiency of the Food Stamp E&T program, 
including: 


e Improving the efficiency of program operations, by conforming work 
registration and sanctioning rules for JOBS and E&T. 


e Making the services provided to program participants more 
comprehensive and better designed to promote self-sufficiency for 
households receiving Food Stamps. 


e Encouraging participants to follow through on their commitments by 
increasing staff follow-up and strengthening the sanctioning process. 


South Dakota used the state’s Family Independence Program (JOBS) as a model 
in designing its E&T demonstration but did not consolidate the delivery of JOBS and 
E&T services as part of the demonstration. Although conformance between E&T and 
JOBS was achieved on most program features, the demonstration design deviated from 
the JOBS program in several respects: 

e Individuals participating in self-initiated training were considered 


exempt from participation and were not eligible for supportive services 
as volunteers. 


e The demonstration did not encourage participation by volunteers. 


e Because of cost constraints, the E&T demonstration did not include the 
implementation of a full-fledged case management system, e.g., through 
the assignment of a DSS case manager for every E&T participant, as is 
the case with JOBS. 


CONTEXT 

E&T Services Prior to the Demonstration 

Prior to the E&T/JOBS Conformance Demonstration, DSS contracted with the 
South Dakota Department of Labor for the delivery of one or both of the following 
E&T activities to all mandatory work registrants through its local Job Service offices: 


¢ Job search. Individuals assigned to job search were required to 
completed up to 12 employer contacts per month and could be required 
to conduct two months of job search back to back or during different 
seasons within a twelve-month period, depending on job availability in 
the local labor market. 


e Job readiness training. This component varied from field office to field 
office in duration, frequency of scheduling, detailed content, and 
delivery arrangements. 

There was no prescribed sequence of participation in these activities. Each local 
office had the flexibility to design the components and assign participants to them based 
on the local labor market and the situation of the individual participant. The assigned 
E&T staff person in the local Job Service offices often had other responsibilities in 
addition to the E&T program. In a number of cases, the same individual was 
responsible for serving both E&T and JOBS clients, but the two programs were 
considered distinct and staff followed separate procedures manuals developed for each 
program. 


Information provided by South Dakota’s local impact evaluator and referred to 
the state’s process and implementation report showed that, of 2,014 individuals who 
received E&T services between July 1, 1990 and June 30, 1991, 89 percent 
participated in the job search component and 28 percent participated in job readiness 
training. (Seventeen percent participated in both components.) 


Shifts in Non-Demonstration E&T Services During the 
Demonstration Period 


South Dakota transformed its entire E&T operations as part of the demonstration. 
There were no counties operating “non-demonstration” E&T services after the start-up 
of the demonstration. 
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DEMONSTRATION SERVICE DESIGN 


Initiation of the Demonstration and Evolution of Demonstration 
Services Over Time 


In March 1993, DOL field staff were instructed to phase out the old E&T 
program in preparation for implementing the demonstration. Demonstration activities 
were Officially implemented effective April 1, 1993. There was a slight delay in 
making the expanded supportive services available to demonstration participants; these 
services became operational in July 1993. In April 1994, community service was 
added as a new demonstration component activity. (Prior to that time the community 
service component could not be offered to demonstration participants, because the state 
did not have Workers’ Compensation insurance coverage for these individuals.) 
Throughout the demonstration period, the planned design was implemented as intended 
without major changes over time. 


Targeting and Participation Patterns 


Like other demonstration states, South Dakota implemented the JOBS work 
registration rules for demonstration participants. Because the JOBS program in South 
Dakota requires participation by all caretakers with children who are at least 1 year of 
age, the inclusion of caretakers responsible for children between 1 and 6 was expected 
to have a major impact on the size of the work registrant pool for the demonstration 
and the characteristics of work registrants. As documented in the South Dakota 
process and implementation report, the number of mandatory work registrants in the 
20-county project area increased over 30 percent from 4,696 to 6,151 between FFY 92 
and FFY 94. The report did not provide information on the extent to which this 
increase resulted from the new work registration rules. 


South Dakota selected priority target group criteria that mirrored the JOBS 
priority target groups. In order of priority, the demonstration targeted work registrants 
who: 

e Were under age 20 without high school completion or equivalency 

certificate. 


e Were between ages 20 and 24 without high school completion or 
equivalency certificate. 


e Were between ages 20 and 24 with little or no work experience during 
the past twelve months. 


e Received Food Stamp assistance during 36 out of the last 60 months. 


¢ Were pending Food Stamp approval and had received Food Stamp 
assistance during 36 out of the last 60 months. 


Information provided in the South Dakota process and implementation report 
Shows that target group members accounted for approximately 40 percent of all 
mandatory work registrants. Field staff were instructed to begin calling in mandatory 
work registrants from target groups in priority order. Although volunteers (individuals 
exempt from work registration) were eligible for demonstration participation, 
participation by volunteers was not encouraged in South Dakota. Once all mandatory 
target group members had been called in, staff were instructed to begin calling in all 
other mandatory work registrants. South Dakota hoped that its E&T contractor would 
be able to continue to reach a high percentage of the total mandatory work registrant 
pool. 


Data provided by South Dakota’s local impact evaluator indicates that, of a total 
of 9,779 individuals designated as mandatory work registrants between July 1993 and 
June 1994, a total of 1,027 (10 percent) participated in demonstration services. ! Only a 
few participants were categorized as exempt volunteers. Among the 1,027 participants, 
37 percent were males and 67 percent were females. Although target group members 
received priority for demonstration services, representation from non-target group 
mandatory work registrants increased over time, as the JSRs began to increase their 
caseloads. During FFY 1995, the ratio of target group to non-target group members 
averaged almost exactly 50 percent/50 percent. 


Service Design 

Under the demonstration, the South Dakota Department of Labor remains the 
primary provider of services to E&T participants, under contract to DSS. In the Sioux 
Falls service area, DSS also contracted with a second provider to serve mandatory 
work registrants from non-target groups, since the Job Service office in this site did not 
have the capacity to serve all mandatory work registrants. After receiving information 
from DSS on new mandatory work registrants (including their priority target group 
status), E&T staff within the local Job Service offices are responsible for calling in and 
serving demonstration participants. 





1A geregate statistics reported by the state to FCS for federal FY 1994 indicated that 1,734 
individuals out of 8,611 mandatory work registrants over the year (or 20 percent) had participated in 
E&T services. As defined by FCS, however, “placement rates” may double count individuals placed in 
more than one E&T service component. 
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The demonstration design substantially expanded the initial assessment and 
employability planning process for E&T participants, as well as calling for ongoing 
contacts between participants and their E&T Job Service Representative (JSR) over 
time. At an initial face-to-face meeting with each participant, the JSR takes a work 
history and assists in the preparation of an individualized employability plan with short- 
and long-term goals. A variety of career interest and aptitude assessments are available 
as needed to assist participants in setting employment goals. JSRs are encouraged to 
assign participants to the service components most appropriate for each participant. 
Under the demonstration, participants remain part of the JSR’s caseload until they are 
deregistered as a result of becoming exempt, or leaving the Food Stamp rolls. JSR’s 
are responsible for tracking participants’ progress through education or training 
programs to which they are referred, as well as monitoring their progress in finding 
and retaining employment. 


Available service components were substantially expanded under the 
demonstration to include: 


e Job search. The content of this component was not changed from the 
pre-demonstration period, but it was no longer presumed to be the most 
appropriate initial service for all participants. 


e Job readiness training. As during the pre-demonstration period, the 
content and delivery arrangements for this component varied from field 
office to field office. 


e Secondary education. Secondary education consists of GED training 
for individuals who have not completed their high school education and 
adult basic education. Referral to secondary education is mandatory for 
individuals under 20 who have not completed high school. In some 
offices participants are referred to JTPA-funded programs for secondary 
education. In other offices, this component is provided through referral 
to adult education programs operated by the public schools. 


e Post-secondary education skill training. Post-secondary education is 
provided through referrals to vocational technical schools, community 
or four-year colleges, or JTPA-funded skill training. 


e On-the-job training. This component is provided through referrals to 
JTPA activities, administered by other staff within the Job Service 
offices. During the final year of the demonstration, the state plan calls 
for the direct funding of on-the-job training with Cooperative 
Agreement funds. 


e Work experience. This component is available through referrals to 
JTPA work experience activities. However, no demonstration 
participants entered this component during the 12 month period from 
July 1993 through June 1994. 


¢ Community service. Beginning in April 1994, Food Stamp E&T 
participants can be required to work 20 hours per week for public or 
private non-profit employers in meaningful job assignments lasting no 
longer than 9 months. This component is administered directly by the 
JSRs. 


Information provided by South Dakota’s local impact evaluator and included in 
the state’s process and implementation report describes the frequency with which 
different service components were utilized during the 12-month period from July 1993 
through June 1994. Under the demonstration, job search was still the most frequently 
used component, utilized by 81 percent of all demonstration participants. Job readiness 
training was received by 23 percent of all participants. The demonstration was 
successful in increasing the percentage of participants receiving education services: 16 
percent of all participants received secondary education, and another 2 percent 
participated in post-secondary education. On-the-job training was received by 2 
percent of all participants. Community service, which had been initiated only a few 
months before the end of the reference period, was received by less than 1 percent of 
all participants. 


Under the demonstration, E&T participants were eligible for an expanded menu 
of supportive services, including relocation assistance, reimbursement of job search or 
training travel expenses, employment-related clothing or tools, minor auto repairs, etc. 
However, these payments were made available and approved only in response to 
individual participant requests. In practice, JSRs did not widely advertise the 
availability of these services and only occasionally approved supportive service 
payments for E&T participants. Thus, supportive service expenditures actually shrank 
under the demonstration as compared to the previous E&T program (which had 
automatically provided $25 transportation allowances to all participants who completed 
required job search activities.) 


Reimbursement of child care expenses was available for demonstration 
participants only for short-term or temporary child care needs. For longer term needs, 
participants were referred to the state’s block grant-funded child care program. 
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Evolution of Demonstration Design Over Time 

The demonstration design was substantially stable over time. Minor changes, 
already noted, included the implementation of community service as a new service 
component available to E&T participants in April of 1994, after a full year of 
demonstration operations. In addition, the plan for the final demonstration year calls 
for the introduction of on-the-job training as a component directly funded with 
Cooperative Agreement funds. It is not year clear whether this component will be used 
for substantial numbers of participants. However, the amount available for this 
purpose ($200,000) is significant. 


PROGRAM COSTS 

Because South Dakota has transformed its entire statewide E&T program 
operations as a result of the demonstration, demonstration operations have been 
supported through a combination of the 100 percent federal formula E&T allocation, 
additional expenditures for E&T operations with 50 percent federal/50 percent state 
funds, and Cooperative Agreement funds received to support the demonstration and its 
evaluation. During the demonstration period, supportive services to E&T participants 


have been provided using Cooperative Agreement funds. The expenditures reported by 
the South Dakota Department of Social Services for FY 1993 through FY 1995 and 
planned for FY 1996 are summarized below. 























100% Formula | 50% State - Cooperative Total 
Funds 50% Federal Agreement 
Funds Funds 


BE YS1095 $221,663 $247,163 $ 79,884 $ 548,710 
FFY 1994 $203,379 $338 ,662 $ 84,555 $ 626,596 
EE Yyele5 $203,477 $422,473 $115,123 $ 741,073 


FFY 1996 $199,748 $708,815 $320,438 $1,229,001 
(planned) 


Program operations did not begin until the middle of FFY 1993 and full 
caseloads were probably not reached until the beginning of FFY 1994. Increased 






program expenditures over time have reflected the need for additional staff time to 
provide the more intensive assessment and service planning provided to each individual 
E&T participant. As a result staffing requirements were increased in the Department 
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of Labor’s contract and a new contract was established with the Career Learning 


Center in the Sioux Falls area. 


SUMMARY OF KEY DEMONSTRATION FEATURES 

The key program features tested by the South Dakota Food Stamp E&T/JOBS 
conformance demonstration are the following: (1) an expanded and individualized 
service planning process; (2) an increased emphasis on utilization of education services, 
particularly for individuals without high school completion; (3) little encouragement 
given to voluntary participation; and (4) a continued interest in calling in a high 
percentage of all mandatory work registrants. The demonstration model was used 
statewide for all counties serving E&T participants. 
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APPENDIX E: 
THE TEXAS BOND DEMONSTRATION 
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THE TEXAS BOND DEMONSTRATION 


GOALS AND OBJECTIVES 

The Texas Food Stamp Employment and Training/JOBS Conformance 
Demonstration, known locally as BOND (“Better Opportunities for New Directions”) 
was implemented in a single county between October 1993 and September 1996 to test 
the feasibility and effectiveness of conforming the E&T and JOBS programs. As 
described by the Texas Department of Human Services (DHS),! the objectives of the 
demonstration included: 

e Providing expanded and enhanced activities and support services for 

E&T participants, both through services directly funded with E&T 


dollars and services provided by collaborating education and training 
agencies from other funding streams. 


e Improving participation through a stronger sanctioning policy. 


e Targeting resources based upon participant need, rather than operating a 
“one size fits all” E&T program. 


e Providing continuity of services to E&T and JOBS participants who 
experience a change in program eligibility while participating in 
employment and training services. 

e Increasing program efficiency and reducing program costs through 
common administrative processes, support materials, staff training, and 
a single service delivery system. 


e Assisting participants to move toward self-sufficiency. 


As part of the BOND demonstration, DHS undertook full consolidation of the 
E&T and JOBS programs in the demonstration county. JOBS policies and procedures 
were applied to participants in both programs, staff serving the two client groups were 
merged, case management and service delivery procedures were consolidated, and, 
with few exceptions, identical services were provided to JOBS and E&T participants. 


1 The administration of the Food Stamp Employment and Training Program was transferred to the 
new Texas Workforce Commission, effective June 1, 1996. 
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CONTEXT 

E&T Services Prior to the Demonstration 

Prior to the demonstration, DHS contracted with several different entities for the 
delivery of services in the 56 counties with operating E&T programs. The Texas 
Employment Commission (TEC) held a contract for E&T services in 53 counties, while 
the Texas Association of Private Industry Councils (TAPIC) held a contract for E&T 
service delivery in the remaining 3 counties.? 


Beginning in 1993, DHS implemented a priority targeting and selection 
procedure for the E&T program that sorted work registrants into “Service Levels” 
based on their previous education and work histories. Work registrants categorized as 
Service Level I (high school completion and recent job experience) and as well as 
individuals categorized as Service Level II (some recent work experience and at least 
8th grade completion) were referred to E&T contractors. Work registrants categorized 
as Service Level III (significant employment barriers or less than an 8th grade 
education, limited or no work history, and no job skills training) were not required to 
participate in E&T and were not referred to E&T contractors. 


The E&T contractors were responsible for providing a one-week 20-hour job 
search skills training/job readiness component (available in about half the E&T 
counties) and monitoring a mandatory 30-day job search process that required 24 
documented employer contacts over the 30-day period. There were few instances of 
formal referrals to other providers, although a joint referral form existed for JTPA, 
Food Stamps, and AFDC clients. Clients who had additional needs or who wanted to 
pursue allowable activities other than those provided by the contracted E&T provider 
were given information about other community resources. Clients who used the 
information to enroll in an education or training activity could continue in that activity 
for up to two years if they remained a mandatory FSE&T work registrant. Additional 
E&T components described in the state plan included vocational training, non- 
vocational education, work experience, and refugee social services. After completing 





2 TAPIC also offered Survival Skills Training for Women as an optional E&T component in three 
additional counties. 


3 This targeting procedure was based on the targeting system already in place in the JOBS 
program. 
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the job search component, participants were considered to have fulfilled their E&T 
responsibilities for 12 months. 


Information collected by Texas’s local evaluator for a seven month baseline 
period in FY 1993 prior to start-up of the BOND program in the demonstration county 
indicates that 85 percent of E&T participants received individual job search, 24 percent 
received job search skills/job readiness training, 6 percent participated in adult 
education after enrolling in E&T services, and 2 percent received survival or life skills 
training. 


Shifts in Non-Demonstration E&T Services During the 
Demonstration Period 


By the time the demonstration was implemented, the statewide E&T program was 
evolving towards the JOBS model. Beginning in FY 1994, DHS consolidated its 
service delivery arrangements for E&T and JOBS services through an agreement with 
the Texas Employment Commission. As part of the redesigned E&T service model, 
E&T components provided by TEC in all 56 counties include: 


e Job search skills training/job readiness training. A two-week 40-hour 
job preparation seminar. 


e Life skills training. A two-week 40-hour survival skills training 
component, introduced by TEC in a limited number of counties in 
FY 1994, and available in all 56 E&T counties in FY 1995. 


e Job search. A six-week component that requires 10 documented 
employer contacts per week, supplemented by a weekly group 
employment seminar that meets for two to three hours per week to 
review job contacts and offer support in job search skills. 


All participants for whom the above services are appropriate are now expected to 
complete the entire sequence of services or enter employment as soon as feasible. In 
case of limited service capacity, individuals categorized as Service Level I are to be 
given priority for the job search skills/life skills seminars. Thus, since FY 1994, all 
E&T participants throughout the state have been offered a sequence of intensified 
preemployment training services and job search supports that is substantially different 
from the independent job search component that had been the norm for E&T 
participants prior to FY 1993. By FY 1995, what had previously been at most a four 
or five week sequence of E&T services had been extended to a ten week sequence of 
services. As a result of the greater intensity of the transformed service design, many 
E&T offices no longer had the capacity to serve all mandatory work registrants. By 
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FY 1995, many offices had reduced the number of work registrants called in for 
services to the number needed to fill the available service slots in the preemployment 


and group employment seminars. 


Additional E&T components still available through non-reimbursable referrals to other 


community providers include: 


e Vocational training, provided by JTPA or local education providers. 


e Non-vocational education, including basic literacy training, GED, 
remedial education, post-secondary education, and ESL instruction, 
through non-reimbursable referrals to JTPA and local education 
providers. 


¢ Work experience training, available through JTPA or other community 
programs. 


e Refugee social services, including ESL, employment services, and 
vocational training, available through federal funding in eight counties 
with high refugee populations. 

Participation in these education and training services continue to be limited to 

individuals who are interested in and pursue these services. 


Beginning in FY 1994, DHS increased the transportation allowances available to 
E&T participants in an effort to conform transportation payments statewide between 
E&T and JOBS. Expenditures over $25 per month are funded solely with state 
moneys. By FY 1995, the same transportation assistance policies and payment 
schedules were applied to E&T and JOBS participants. As described below, under 
evolution of the demonstration design over time, the required sequence of EXT 
services was amended in October 1995 as part of an overall state welfare reform 
initiative that emphasized the responsibility of work registrants to seek immediate 
employment. 


DEMONSTRATION SERVICE DESIGN AND OPERATION 

Initiation of the BOND Demonstration 

DHS chose to implement the BOND demonstration in a single E&T county. 
McLennan County was selected as the site of the demonstration because it had an E&T 
program of intermediate size, a successful JOBS program, and sufficient local 


education, training, and transportation systems to support the project. McLennan 
County accounted for about 1.8 percent of the work registrants in all E&T counties. A 
small demonstration project area was felt to be appropriate, given the significantly 
higher costs of the JOBS program to which E&T operations in the demonstration area 
were intended to conform. 


Demonstration planning occurred during the first year of the federal conformance 
demonstration grant. Delivery of regular E&T activities in the demonstration county 
was curtailed in August 1993 and discontinued in September 1993 to free staff for 
training. The BOND demonstration was implemented on October 1, 1993. 


Targeting and Participation Patterns 

As a result of replacing the E&T work registration and exemption requirements 
with the JOBS exemption criteria, the BOND program was expected to reach new 
groups of mandatory work registrants, including caretakers responsible for children 
ages 3 to 5, custodial parents under age 20 who had not completed high school, 
students enrolled only half-time, Food Stamp recipients receiving or applying for UI 
benefits, and individuals participating in substance abuse rehabilitation programs. As 
documented in the Texas process and implementation report (p. 28), the number of 
mandatory work registrants in McLennan County increased 32 percent from 2,814 to 
3,716 between March 1993 and March 1994. The report did not provide information 
on the extent to which this increase resulted from the new work registration rules; 
however, statewide work registrant rolls increased by 29 percent during the same 
period, suggesting that non-demonstration-related factors were also at play. 


Like the E&T program, the BOND demonstration used Service Level 
assignments to determine which work registrants would be required to participate in 
services. However, whereas an individual categorized as Service Level III had no 
access to E&T services, Service Level III clients could volunteer for BOND services 
(although they were usually referred to community resources). Beyond targeting 
Service Level I and II customers, the BOND demonstration did not establish any 


4 Although McLennan County had a well-developed education and training system, its JOBS 
program was not necessarily typical of other counties, because: (1) it lacked well-developed coordination 
agreements with the JTPA service delivery system at the outset of the demonstration, (2) it had an 
atypically high utilization of post-secondary training by students in self-initiated training, and (3) an 
unusually high number of JOBS participants were high-school students enrolled in a local drop-out 
program. These features influenced the service patterns for the E&T participants in the demonstration. 
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additional priority target groups within the E&T work registrant population to mirror 
the JOBS priority target groups (e.g., individuals under age 24 who had not completed 
high school; individuals who had received Food Stamp benefits over a long time 
period). 


The participation levels in the BOND program were constrained by the staff- 
intensive nature of the services offered to program participants (as described below, 
these services consisted of group or individual case management by DHS staff and 
participation in a sequence of job readiness/job search activities and/or long-term 
education and training). Individuals were called in or invited to participate in services 
as program openings became available in individual case manager caseloads—for 
individuals interested in or appropriate for long-term education or training—or in job 
preparation seminars—for more job ready individuals. Participation data summarized 
in the Texas process and implementation report suggests that participation rates for 
BOND were low. For example, the 188 individuals participating in BOND during an 
average month in FY 1994 (p. 85) would have comprised only 5 percent of the total 
work registrants in McLennan County in March 1994 (p. 82). 


JOBS volunteer policy was applied to the demonstration. Exempt and non- 
exempt E&T work registrants could volunteer and received priority for BOND 
services. During the eligibility certification process, DHS intake workers gave eligible 
Food Stamp recipients the opportunity to volunteer for BOND participation. In 
addition, volunteers could “walk-in” and request services from the BOND program at 
any time. The Texas process and implementation report says that, according to 
Management reports prepared by DHS, between 30 and 40 percent of BOND 
participants were exempt or nonexempt volunteers (p. 56).5 


Service Design 

Among clients categorized as Service Level I and II, assignments to BOND 
services were Closely related to service level assignment. Service Level I clients were 
normally referred to the job search and job readiness components operated by TEC. 
As noted above, the enhanced job readiness and job search components available to 
BOND participants—including a 40-hour job preparation seminar, a 40-hour 





> This may include mandatory work registrants who volunteered for BOND services prior to being 
called in as well as exempt volunteers. 
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survival/life skills seminar, and a 6-week individual job search activity in combination 
with a weekly group employment seminar—were identical to the components available 
to E&T participants statewide under the redesigned contract with TEC. Under the 
BOND demonstration, participants in TEC services received group case management; 
group case managers are responsible for tracking attendance in assigned services, 
monitoring client progress, and identifying non-compliance situations. 


In BOND, Service Level II clients were usually referred to and enrolled in 
components that addressed their education or social skills deficits, such as adult 
education or special survival skills training. Service Level I clients could also 
volunteer for assistance with self-initiated post-secondary training plans. All 
participants in education and training services received individual case management 
services from a DHS case manager. Available services included: adult education 
through referral to GED, ABE, or ESL instruction in local adult education programs; 
supports to at-risk high school students through a program called “communities in 
schools;” post-secondary training through provision of case management and support 
services to support attendance at local technical or community colleges; job skills 
training through referral to local JIPA programs. Additional activities available to 
BOND participants included unpaid work experience and on-the-job training. 


Child care assistance for caretakers of young children was available through the 
BOND demonstration. The demand for child care was high, and availability of child 
care was described in the Texas Process and Impact Evaluation as a potential “magnet” 
for program participation. Child care utilization was concentrated among BOND 
participants attending post-secondary education or training. As described above, 
statewide E&T transportation payments had been established at a level equivalent to 


that available in JOBS starting in FY 1994. 


Information included in the Texas process and implementation report (p. 68) 
indicates that during the first year of BOND operation, 45 percent of participants 
received survival/life skills training, 27 percent received job preparation/job search 
skills training, and 25 percent participated in individual and group job search activities. 
First-year statistics for education and training components showed that 8 percent of 
participants received adult education services, 11 percent participated in the high- 
school-based “community in schools” program, and 35 percent participated in post- 
secondary education. During the second year of program operation, the number of 
individuals participating in adult education and post-secondary training increased 


substantially (from 8 percent to 13 percent in adult education and from 35 percent to 44 
percent in post-secondary education), accounting for an increased proportion of all 
BOND participants in education and post-secondary training. 


Evolution of Demonstration Design Over Time 

The demonstration design implemented in McLennan County was quite stable 
between demonstration startup in October of 1993 and October 1995. However, in 
October 1995, a dramatic redesign of welfare-to-work programs was implemented, as 
part of a new Texas “Personal Responsibility and Work First” initiative. Under this 
initiative—which affected the design of JOBS and E&T programs, in addition to the 
BOND demonstration—work registrants are required to participate in a redesigned 
sequence of services that emphasizes the responsibility of the participant to seek 
immediate employment. An initial four days of job preparation training are followed 
by three weeks of job search, followed by another four days of job preparation 
training, followed by another three weeks of job search. The state’s process and 
implementation Study covered FFY 1994 and FFY 1995; it did not document the 
redesigned program that was in place after October 1, 1995. 


PROGRAM Costs 

During FY 1994, the first year of BOND implementation, the E&T program in 
McLennan County accounted for 1.8 percent of all Food Stamp work registrants in the 
state and consumed 3.7 percent of the statewide E&T budget ($586,879 out of the state 
budget of $15 990,276). Texas made the decision to fully fund the E&T demonstration 
from the existing E&T budget. Child care expenditures were unexpectedly high, 
accounting for over 35 percent of the total BOND budget. Cooperative Agreement 
funds, totaling $586,365, were used to support the evaluation of the demonstration. 


SUMMARY OF KEY DEMONSTRATION FEATURES 

The BOND demonstration is characterized by: (1) full consolidation of E&T and 
JOBS operations; (2) a high level of participation by exempt volunteers; (3) the 
development of a two-tier service design offering an increase in the intensity of job 
readiness and job search services for more job ready participants and access to basic 
and post-secondary education and training for individuals with more serious 
employment barriers; and (4) a Significant investment in child care expenditures to 
support attendance in long-term education and training. 
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Summary of Evaluation of the E&T/JOBS Conformance Demonstrations: Final Synthesis of 
Implementation and Process Evaluations, and Evaluation of the E&T/JOBS Conformance 
Demonstrations: Final Synthesis of Impact and Cost Evaluations 


From October 1, 1993 to September 30, 1996, the Food and Nutrition Service of the U.S. 
Department of Agriculture sponsored demonstration projects in Georgia, Hawaii, Missouri, 
South Dakota, and Texas to demonstrate the feasibility and effectiveness of operating the Food 
Stamp Employment and Training (E&T) program under the same legislative and regulatory 
terms as the Job Opportunities and Basic Skills (JOBS) program for Aid to Families with 
Dependent Children (AFDC) recipients. Common objectives of the demonstrations were to 
increase compliance with E&T participation requirements among mandatory work registrants, 
target services to individuals most at risk of long-term dependency and those most likely to 
benefit from E&T services, improve participant outcomes, and improve the cost efficiency of 
welfare to work services. 


States conformed their E&T programs to the JOBS model by changing their participation rules 
and E&T service designs. Each state was also required to design and arrange for an independent 
evaluation of its demonstration. The first volume of this report synthesizes the implementation 
and process evaluations of the demonstrations in the states. The second volume is a synthesis of 
the states’ impact and cost evaluations. 


The implementation and process evaluations focus primarily on the objectives of the 
demonstration and how the states designed their programs. Outcomes are covered in the impact 
and cost evaluations. These impact evaluations provide evidence of at most modest 
improvements in participant outcomes. Some of the state findings include: 


e Client targeting criteria and service approaches established for poor households with 
dependent children do not automatically make sense when transferred to the Food 
Stamp Program, since the demographics and participation dynamics of Food Stamp 
recipients can vary significantly from those of the AFDC population. 


e Intensive services were provided to a limited number of work registrants under the 
demonstration. It is unclear how the provision of these services would have affected 
outcomes for those work registrants who did not receive them, but probably would 
have under the regular Food Stamp E&T Program. 


e Selective targeting to priority groups can increase services to these groups. However, 
providing supportive services such as child care assistance to E&T participants may 
entice them to stay on the Food Stamp Program longer. 


The experiences from the demonstration states suggest that regulatory conformance and 
administrative consolidation between E&T and JOBS is administratively feasible at the state and 
local level. However, the findings described above may cast doubt as to whether such 
conformance improves outcomes for E&T participants. Moreover, the world of welfare to work 
has changed since these demonstrations were first authorized. Welfare reform replaced the 
AFDC program with state Temporary Assistance for Needy Families block grants and brought 
about significant changes to the Food Stamp E&T program. 
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FOREWORD 


From FY 1993 through FY 1996, the Food and Nutrition Service of the U.S. 
Department of Agriculture supported five state demonstration projects designed to test 
the feasibility and effectiveness of operating the Food Stamp Employment and Training 
(E&T) program under the same legislative and regulatory terms as the Job 
Opportunities and Basic Skills OBS) program for AFDC recipients. Each of the 
participating states was responsible for designing and arranging for an independent 
evaluation of its demonstration. Social Policy Research Associates (SPR) was selected 
by the Department of Agriculture to assist the demonstration states and their evaluation 
contractors in their evaluation efforts. As the national evaluation contractor, SPR was 
responsible for completing a critical review of each state’s evaluation design and 
providing ongoing technical assistance to the state evaluators in collecting and analyzing 
data, interpreting study findings, and preparing written evaluation reports. We were 
also charged with preparing a synthesis of state evaluation findings. 


This volume synthesizes the findings from the states’ impact and cost evaluations. 
Conclusions from the states’ implementation and process evaluations are synthesized in 
a companion volume (Kogan and D’Amico, 1997). The individual state-level impact 
and cost and evaluation reports that presented the findings summarized in this volume 


are noted as references herein. 


Over the course of the four-year demonstration period, SPR staff benefited from 
site visits to each of the demonstration states and from many telephone and written 
communications with the state evaluators and the state program staff responsible for the 
E&T/JOBS conformance demonstrations. We would like to express our appreciation 
and gratitude for the full cooperation of these individuals in making the state evaluation 
designs as consistent as possible (within the constraints established by widely varying 
demonstration designs and data sources), so that the state evaluations could support the 
national evaluation synthesis. We also acknowledge the hard work of the states’ 
evaluators without which this report could not have been prepared. 


We would also like to express our appreciation to Boyd Kowal, Barbara Murphy, 
and Christine Kissmer — our government technical representatives within the Food and 
Nutrition Service’s Office of Analysis and Evaluation over the course of the project — 


for their support and encouragement. 





TABLE OF CONTENTS 


EXECUTIVE'SUMMAR Yee D8. Wi US Sener is cre range Tors otras. ES-1 

I. CONFORMANCE BETWEEN E&T AND JOBS: AN OVERVIEW ................ I-1 

HISTORY AND OVERVIEW OF THE DEMONSTRATION .........0.0c0ceceseceeeeeeeeeeeens I-1 

OVER VIEWOFR THE REPORT 4 Leet | RINE ITAA I-8 

Il. OVERVIEW OF DEMONSTRATION EVALUATION DESIGNS................. II-1 
SUMMARY OF THE DEMONSTRATIONS’ DESIGN AND 

DAPLEME NICATION Meee te ee eM as her eso gunn gclna doe ¢ « lv BPs Sige II-1 

KEY, FEATURES OF SLATE. DEMONSTRATION DESIGNS... 5:.:00csc-fteeece ort nee. II-3 

EVALUATION EG IGiy Gatun ott macgmn re Cn SeROmen Cte enn nN Ga nl, Seem ing ae II-4 

COMDATISON SLCSiteet Mien ctor te RN Se ny yn NR satan II-5 

W iLife Ie RANG Gas. SS1 SOIC Utena ee eee on och eee a II-6 

BY CER OSE OMIDALI SOD, gett ee cca act OTe EM scte Bho sce ket II-6 

CombinedsPre/ Posmand Compalisou slices. eeu nieesudnct tk cn eae. II-7 

Evaluation" Samplessran brah de cookin. al nen Mee aati. ek. lob eee oe II-8 

SUMMARYS fac cees ae ett cey oie tas TREES Met AMOUR CCRT Neetu oes MEMNNR. 6x0 ss Shee dads II-9 

Il. ESTIMATED IMPACTS OF THE DEMONSTRATIONS ....................... HI-1 

PARTICIPATIONS PA ETERNG egcar eettea. etn crear aa ce moet ten eng ae es III-1 

Mandatonvaw Orc RCOISLI Al (Sacer ete ieee ere ere eee one ee ee II-1 

PartiCipatiGnaR alesis matin on atin ny ae eat oaths Vi ve ne avd 02 III-2 

Ni ONINC CCL Seren tert Cs ter ne ene Se See ge Rs ain cae Sek ae Ill-4 

Priority Groups and Participant Characteristics ...................eeeeeee eee III-5 

SADCUIONING rete ee a ete Mees aCe re rn Steuer eet ant a7; II-7 

ESCISSER V UGE Mr ore oper Mrent ee Siem career se nocd eases scree et wade IlI-8 

EV DESTOLIOCLVICCS tense, rae Sor Mest ch tet nee tem res ea ieee Sennen ae IiI-8 

TRCETISICV CO SOCT VICES eee ere tet ee, ernie eta anc oe IlJ-10 

PPAR ORMVIARKETZOUTCOMESE tortsccr tty etttn ts cesar aren cater Cooter NS lecccnnl III-10 

FOODS VAMP DENEEI I RECELP lerieirats rr or tre sites tates ccs horde ee nee eee IlI-12 

(COSTS ee ae eee ty ae, re te teeta remo o nade tas Shee a I-12 


ill 


TV) CONCLUSIONS 9. tics tates steerer erreatiart immerse acta here IV-1 
INTRODUCTION toscee sect eee ee ane ete iateade cig cg IV-1 
INCREASE E&T PARTICIPATION BY MANDATORY WORK 
REGISTRANTS TARGETED FOR SERVICES .....--+:eereeeressenernsss esses eee IV-1 
TARGET SERVICES TO HIGH RISK AND MOTIVATED FooD STAMP 
RECIPIENTS sie Ld dO, LEA TRE IE Te IP RPT IV-3 
MATCH SERVICES TO PARTICIPANTS’ NEEDS AND IMPROVE 
PARTICIPANT OUTCOMES 00.0.2. otnnieus angeecranpensaperzdagerccee* 2 oto Oss a IV-5 
IMPROVE THE COST-EFFICIENCY OF WELFARE-TO-WORK SERVICES ...---++++++++ IV-6 
LESSONS LEARNED FROM THE CONFORMANCE DEMONSTRATIONS ....--++++++05° IV-6 

APPENDICES: 

Appendix A: The Evaluation of the Georgia JET Demonstration ...........-:ssesseee A-1 


Appendix B: The Evaluation of the Hawaii Conformance 
Demonstration /s:::cctsccsietheta de veka snes AMMO CTIEAGS oh PREP SEE ++ e+ 7° 


Appendix C: The Evaluation of the Missouri JET Demonstration 
ProjeClehisy:scscstisdee feces ee ee ae 


Appendix D: The Evaluation of the South Dakota Family 
Independence Food Stamp Employment and 
Training/Jobs Conformance Demonstration ..........0:2seeeeeeeeeeeenre ee 


Appendix E: The Evaluation Of The Texas BOND Demonstration 
ats) co MORE eS eating og Eno hs tienes oi i AiR aaa 


REFERENCES sa aeitens vecsecch ccc clas oa osc ck aa aaa ae 


iV 





Table II-1 

Table III-1 
Table III-2 
Table III-3 


Table III-4 


Table III-5 
Table II-6 
Table III-7 
Table III-8 


Table III-9 

Table II-10 
Table IHI-11 
Table I-12 


LIST OF TABLES 


Kevaleatiires: Om pemonstrdi luis aint cower at steers. ane sees II-2 
Estimated Impacts on Work Registration .................:ccceeeeene enone III-2 
Estimated Impacts on Participation Rates .....:1.--0.0¢2:.-1-0) tf ee eee II-3 
Estimated Impacts on Service to Volunteers, Otherwise Execpt 
(Rercentace Ol Pallucipants). ete tira. ucnck see Om ee III-4 
Estimated Impacts on Participant Characteristics 

(REPCenlaDGreOUlS jane emer Rar rence ec man cece res pete. terre t rs III-6 
ESUMmatcd sn DaCts Nn SalIChOUS ee weet te ene ee eerie ro, IlI-7 
Estimated Impacts on Receipt of Services (Percentage Points)......... III-9 
Estimated Impacts on Employment and Earnings....................... IiI-11 
Aggregate Costs During and Before the 

Demonstravion «Annualized re tee eee ce a eesti edene ck III-13 
Cost Per Participant During and Before the Demonstration .......... III-14 
Percentages Distribution. of COSts sate, aden ates one -cnaut toe oe: Ses [I-14 
Wistributionh @ostsiPen Participant = wem ees. ce ees aes. wo ye III-16 
BundsskeveracedibysDemionsttatio tis tech tonte cat ay eae san ies sind I-17 


= 





EXECUTIVE SUMMARY 


From October 1, 1993 through September 30, 1996, the Food and Consumer 
Service of the U.S. Department of Agriculture supported demonstration projects in five 
states—Georgia, Hawaii, Missouri, South Dakota, and Texas—to test the feasibility and 
effectiveness of operating the Food Stamp Employment and Training (E&T) program 
under the same legislative and regulatory terms as the Job Opportunities and Basic 
Skills (JOBS) program for AFDC recipients. Common objectives of the demonstrations 
included: 


e Increase compliance with E&T participation requirements among 
mandatory work registrants and slow down the “revolving door” of 
curing adverse action notices by implementing more stringent 
sanctioning rules. 


e Target services to individuals most at risk of long-term dependency and 
those most likely to benefit from E&T services through selective 
targeting using priority categories modeled after those used in the JOBS 
program. 


e Improve participant outcomes by assessing participants at the outset and 
offering individualized service assignments and service sequences 
matched to participants’ needs. 


e Improve the cost efficiency of welfare-to-work services by streamlining 
the administration of services provided to recipients of different 
assistance programs, such as AFDC, Food Stamps, and state or county 
general assistance. 


To conform their E&T programs to the JOBS model, states made a variety of 
changes to participation rules, including requiring mandatory work registration by 
members of new groups (primarily caretakers responsible for children between 3 and 6 
years of age and recipients of unemployment benefits), giving priority to members of 
groups expected to be at risk of long-term dependency, implementing tougher 
sanctioning policies, and encouraging participation by volunteers. To conform their 
E&T service designs to the JOBS model, states undertook to: provide assessment and 
individual service planning to participants; increase the range of education and training 
services available to participants through nonreimbursable coordination linkages and/or 
direct purchase of enhanced services; require participation in educational components 
by enrollees who had not completed high school; and offer more generous payment 
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schedules for reimbursement of transportation, child care, and other expenses 


associated with participation in education and training activities. 


This Synthesis of Impact and Cost Evaluations describes the evaluation designs 
used by the states and their local independent evaluators and summarizes the findings 


from their impact and cost evaluations. 


EVALUATION DESIGNS 

Evaluation Designs 

The key evaluation design decision faced by each state was choosing a 
comparison method. To conduct a valid impact evaluation, states needed to develop a 
method of estimating what would have happened in the absence of the demonstration. 
In this way, the impact of the demonstration could be derived by comparing outcomes 
achieved in the demonstration to what outcomes would have been had there been no 
demonstration. There are four comparison strategies that states used. 


e Pre/post. Only one state, South Dakota, chose to rely on a pre/post 
comparison for its evaluation. Pre/post comparisons automatically 
control for the unique features of the demonstration site, but risk 
confounding the estimated effects of the demonstration with the effects 
of other changes that occur during the evaluation period. 


e Comparison site. Two states, Georgia and Missouri, relied on a 
comparison site design to evaluate their conformance demonstrations. 
Comparison site designs control for changes over time that affect both 
the demonstration and comparison sites, but rely on the assumption that 
the introduction of the demonstration is the only difference between the 
demonstration and comparison sites that has substantial effects on the 
outcomes being examined. To help assure that the demonstration and 
comparison sites were similar, Georgia selected a matched comparison 
county for each of the counties included in the demonstration. 


e Combined pre/post and comparison site. Two states, Texas and 
Hawaii, used a combined pre/post, comparison site strategy for 
estimating the effects of the demonstration. The strength of the 
combined design is that it adjusts both for preexisting differences 
between the demonstration and comparison site and for trends over time 
that are common to both sites. Both states, however, were unable to use 
the full design for all outcomes because information on some outcomes 
was not available in the predemonstration period. For these outcomes 
they used a comparison site design. 
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e Within-site random assignment. Georgia also implemented a within- 
site, random assignment design to determine the effects of increased 
supportive services. This approach has substantial statistical advantages 
over the alternatives because the random assignment itself assures that 
the demonstration and comparison samples are statistically equivalent. 
This approach was not generally used for the demonstration evaluations 
because the demonstrations were examining the effects of systemic 
changes that were difficult or impossible to limit to a random subset of 
participants. 


When data were available, the states used multivariate procedures, such as 
regression analysis, to adjust for differences in participant characteristics and economic 
condition between the demonstration and the comparison site or period. However, the 
use of these procedures was limited by data availability and was not used for all 


outcomes examined. 


KEY FINDINGS FROM THE IMPACT AND COST EVALUATIONS 


The impact and cost evaluations examined participation patterns (including work 
registration, participation rates, service to volunteers and service to priority groups), 
services (including frequency of types of services and the intensity of services), 
outcomes (including employment, earnings, and Food Stamp receipt), and costs. The 
findings from the evaluations are summarized below. 


Participation Patterns 

The key elements of the demonstrations that were expected to affect participation 
patterns include the use of more inclusive work registration requirements, targeting 
services to groups most likely to benefit from E&T services, encouraging volunteers, 
and implementing stricter sanctioning procedures. Findings of the state evaluations 


related to participation patterns include: 


e Replacing E&T work registration rules with JOBS work registration 
rules seems to have increased the number of mandatory work registrants 
in four of the five demonstration states. Many of the additional work 
registrants were young women with dependent children. 


e Selective targeting seems to have been an effective tool in increasing the 
representation of priority groups among E&T participants. However, 
the absolute number of E&T participants belonging to priority groups 
fell in several states because the decline in overall participation rates 
overwhelmed the effect of selective targeting. 
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e Encouraging volunteers led to substantial participation by volunteers in 
most states. Volunteers tended to be more educated individuals who 
were interested in advancing their education and training. Thus, 
recruiting volunteers seemed to decrease the percentage of participants 
who were high school dropouts and increase the percentage who were 
white. 


e Selective targeting and encouraging volunteers had countervailing 
effects on participant characteristics. While selective targeting was 
designed to focus services on the most needy groups, serving volunteers 
tended to focus services on more educated individuals. 


e Tougher sanctioning procedures led to a lower rate of requested or 
applied sanctions in the three states that implemented such procedures. 
Data from one state suggest that the lower sanction rate may have 
resulted in part from increased compliance. 


Services 

The state impact evaluations confirm that the demonstrations increased the range 
and intensity of education and training services provided to E&T participants. 
Specifically, the impact evaluation results confirm that the demonstration projects 
increased the frequency with which E&T participants received assessment, education 
and vocational training services, and participated in work experience or community 
service activities. The frequency of utilization of individual job search and job search 
skills training declined, as these services began to be matched to individual participants, 


rather than assigned to all participants as a required first service. 


Outcomes 

Despite the overall success in implementing more intensive services, the state 
impact evaluations provide evidence of at most modest improvements in participant 
outcomes. Three of the five states showed increased employment among demonstration 
participants. Because these analyses did not adjust for demographic or economic 
differences between the demonstration and the comparison, they are not conclusive. 
Indeed, the remaining two states found no significant employment effect after 
controlling for participant characteristics and economic conditions, even though there 
were positive effects in simple difference in means analyses. Further, the effects of the 
sharply reduced participation rates in many states may well have offset these modest 
effects. That is, the demonstrations may have had negative effects on the many work 
registrants who did not receive services under the demonstration but would have 


received services under the nondemonstration E&T service design. 


Social Policy Research Associates ES-4 





Moreover, the two state evaluations that examined Food Stamp receipt found 
mixed effects that, if anything, indicate that the demonstration may have increased Food 
Stamp utilization during the first year after participants entered the E&T program. 


Costs 


The cost evaluations clearly show that overall E&T program costs increased 
dramatically (between 43% and 368%) as a result of the demonstrations and that per 
participant costs increased by even more (between 121% and 976%), due to the 
substantial reduction in participant volumes. Thus, while costs per participant ranged 
from $141 to $458 before the demonstrations, they ranged from $311 to $1,733 during 
the demonstrations. In most states, the greatest increase occurred in support costs. 
The bulk of the increase in support costs was for childcare assistance in the two states 
that provided information on childcare costs. 


LESSONS LEARNED FROM THE CONFORMANCE DEMONSTRATIONS 


As noted in SPR’s Synthesis of Implementation and Process Evaluations, the 
experiences of the demonstration states suggest that regulatory conformance among and 
administrative consolidation of different welfare-to-work programs is administratively 
feasible at both the state and local levels. This bodes well for the potential success of 
states choosing the Simplified Food Stamp Program option of consolidating 
administration of Food Stamps and Temporary Assistance to Needy Families (TANF) 
under the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 
(PRWORA). 


However the state impact evaluations raise serious questions about whether 
conformance—as implemented in the E&T/JOBS Conformance Demonstration—leads to 
improved outcomes for E&T participants: 


e The conformance demonstration evaluation results showed that selective 
targeting to priority groups can increase services to those groups. 
However, they also suggest that providing supportive services, 
including childcare assistance, to individuals in education and training 
activities may induce them to stay on Food Stamps longer, as they 
continue those activities. At this point, one can only speculate about 
whether increased Food Stamp receipt in the short run will be followed 
by reduced dependency in the long run as these individuals complete 
their training and obtain employment. 


e One of the lessons suggested by the E&T/JOBS Conformance 
Demonstration is that client targeting criteria and service approaches 
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established for poor households with dependent children do not 
automatically make sense when transferred to the Food Stamp program 
with its rapid caseload turnover and distinct caseloads that include both 
relatively job-ready individuals and individuals with limited employment 
skills and significant individual and family barriers to self-sufficiency . 
Instead, client targeting, service designs, and policies about priority to 
volunteers should be developed specifically to address the needs and 
goals of the E&T program and its work registrants. 


e The demonstration evaluations did not give clear information about how 
the provision of intensive services to a limited number of work 
registrants affects outcomes for those work registrants who did not 
receive services but probably would have under the non-demonstration 
E&T Program. However, it is likely that to improve overall outcomes 
across the entire work registrant pool, E&T programs will have to 
distribute resources across a larger number of participants than did some 
of the conformance demonstration projects. 


The Food Stamp E&T program faces a number of new challenges in the coming 
months as a result of changes to the Food Stamp and cash assistance systems under 
PRWORA and the Balanced Budget Act of 1997. First, the E&T program faces the 
challenge of serving able bodied adults without dependent children (ABAWDs) who are 
at risk of losing Food Stamp eligibility after three months unless they work at least 20 
hours a week. Additional funds have been appropriated by Congress to ensure that the 
E&T program will be able to assist these at-risk Food Stamp recipients to find 
employment or, if jobs are not available, to perform a workfare activity that will allow 


them to retain eligibility for Food Stamps. 


While consistent with the “work first” approach being emphasized for TANF 
recipients, the targeting of Food Stamp E&T services to ABAWDs is likely to require 
the development of distinct service designs and service delivery arrangements—i.e. 
placing participants into workfare assignments—because of the extremely short 
eligibility time limits for these individuals unless they obtain employment. The 
Balanced Budget Act of 1997 requires that 80% of federal E&T funds must be used to 
serve ABAWDs. Thus, at least in the short run, the emphasis of E&T program designs 
will need to be on providing cost-effective mechanisms for ABAWDs to remain eligible 
for Food Stamp benefits when jobs are not available in the local labor market. 


Over time, however, the E&T program may inherit increased responsibility for 
poor households with dependent children who exhaust their eligibility for cash 
assistance. To the extent that resources become available to target services to these 
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households, states will be called on to develop service designs that are effective in 
enhancing employability for individuals with multiple employment barriers. The 
examples of selective targeting, individualized service planning, and service offerings 
tested under the E&T/JOBS Conformance Demonstration will provide a starting point 
for states interested in using E&T resources to address the long-term employability 
development challenges posed by welfare reform. 
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I. CONFORMANCE BETWEEN E&T AND JOBS: AN OVERVIEW 


HISTORY AND OVERVIEW OF THE DEMONSTRATION 


The Food Stamp Employment and Training (E&T) program, initiated by the Food 
Security Act of 1985 and required to be in operation in all states by April 1, 1987, was 
intended by Congress to increase the employability of program participants by 
“[assisting] members of households participating in the Food Stamp program in gaining 
skills, training, or experience that will increase their ability to obtain regular 
employment.” Until recently the program was allotted only limited federal funding—it 
spent less than $140 million annually—to realize these lofty goals.!_ When distributed 
across the 1.3 million individuals who participate in E&T services in a typical year, the 
program spent only about $100 in federal funds, on average, per participant. Asa 
result of cost-sharing requirements, the states contributed additional funds, accounting 
for another $60 million in expenditures annually, which increased the total spending to 
about $200 million nationally, or about $150 per participant. The Personal 
Responsibility and Work Opportunity Reconciliation Act of 1996 (PRWORA) and the 
Balanced Budget Act of 1997 modified the E&T program, including an increase in 
funding. The period covered by this report, however, ends before passage of 
PRWORA. 


Previous E&T program designs have resulted in large numbers of work 
registrants receiving minimal services as well as the issuance of large numbers of 
sanctions for noncompliance.? Prior to FY 1992, E&T program design and client 
targeting decisions were strongly influenced by federal performance standards requiring 
states to serve at least 50 percent of all mandatory nonexempt work registrants either by 
enrolling them in component services or sanctioning them for failure to comply with 
participation requirements. In response to these federal requirements, most states 


1 The Department of Agriculture provided $75 million annually to the states as 100 percent 
federally-funded formula grants for the administration and operation of E&T services. Additional federal 
funds are available on a dollar-for-dollar matching basis to states that want to spend more on E&T 
operations. The federal government also provided matching funds to states to reimburse participants for 
the costs of child care and transportation/training expenses, within federal cost limits. Supportive service 
expenditures in excess of these limits must be provided using nonfederal funds. 


2 Social Policy Research Associates and SRI International, Study of the Food Stamp Employment 
and Training Program: Operations, Funding, and Coordination. May 1992. 
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targeted E&T services broadly to all mandatory work registrants but offered only 
limited services, primarily job search training and individual job search assistance. 
Furthermore, because sanctioning procedures had no “teeth,” E&T case managers 
spent much of their time issuing and curing sanctions, rather than helping participants 
find jobs.3 Perhaps because of the limited services provided to most participants, a net 
impact study performed during the first year of program operations found that the 
program had failed to achieve any statistically significant improvements in employment 
outcomes for E&T participants, compared to what they would have achieved without 


the program.4 


Thus, at the end of FY 1991, state administrators of the Food Stamp Employment 
and Training Program faced an important turning point. They were eager for ideas 
about how to transform the program from one widely perceived as merely an 
administrative requirement for Food Stamp recipients and a paper-processing nightmare 
for program staff into a program providing meaningful and effective employment 
services. Responding to criticisms of the previous E&T program design, the 
Department of Agriculture opened the door to program redesigns at the state level by 
reducing the required participation rate standard from 50 percent to 10 percent of 


mandatory work registrants, effective FY 1992. 


The Job Opportunities and Basic Skills (JOBS) program for AFDC recipients 
offered a potentially attractive model to states interested in developing new approaches 
for the E&T program. At the time that E&T was facing a turning point, the JOBS 
program was being touted as the answer to improving the employability of AFDC 
recipients. The JOBS program design: 

e Used selective targeting to emphasize serving clients who would 

otherwise be at risk of long-term welfare dependency. 

e Emphasized individualized service planning, rather than a “one size fits 

all” approach. 


e Emphasized the improvement of participant employability through the 
delivery of basic education and vocational training services. 


EN 


3 Participants could cure a sanction merely by indicating their willingness to cooperate with the 
program, without taking meaningful steps to comply. 


4 ABT Associates, Inc., Evaluation of the Food Stamp Employment and Training Program Final 
Report. June 1990. 
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e Built on coordination linkages with basic education and vocational 

training providers to leverage additional public funds on behalf of 
program participants. 

In recognition of the need to test new models for E&T design and operations, The 
Mickey Leland Memorial Domestic Hunger Relief Act of 1991 authorized the Secretary 
of Agriculture to conduct a demonstration on “conforming the Food Stamp E&T and 
JOBS programs in 60 project areas.” On March 27, 1992, the Department of 
Agriculture issued an announcement inviting states to submit proposals for operating 
their E&T program in selected project areas under the same legislative and regulatory 
terms as the JOBS program.* States were also encouraged to develop partnerships 
among different employment and training programs to achieve greater coordination 
between E&T and other programs like the Job Training Partnership Act (JTPA), adult 
basic education, and vocational education. 


To conform E&T participation rules and procedures to JOBS policies and 
practices, states interested in participating in the demonstration were permitted to 
change E&T operations in the following ways: 


e Replace E&T exemption criteria with JOBS exemption criteria. Key 
differences were the inclusion in JOBS mandatory work registration 
rules of individuals responsible for the care of dependent children over 3 
years of age (over 1 year of age at state option), rather than over 6 years 
of age as in E&T requirements, and the inclusion of individuals 
receiving UI benefits. 


e Give priority to participation by volunteers, subject to resource 
availability. The JOBS legislation called for priority both to exempt 
work registrants and mandatory work registrants who volunteered prior 
to being called in for services. One group of volunteers cited in the 
legislation consisted of mandatory or exempt volunteers already enrolled 
in self-initiated training when they entered the program. Individuals in 
self-initiated training approved by JOBS were eligible to receive 
supportive services from the JOBS program. 


e Implement selective targeting of clients, following the state’s JOBS 
example. For the AFDC population served by JOBS, these target 
groups included individuals under age 24 without a high school diploma 


5Although the Department of Agriculture encouraged states to propose demonstrations that tested 
full conformance between the E&T and JOBS programs, states were permitted to submit proposals for 
limited conformity, if they cited barriers that prevented them from guaranteeing full conformance. 
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or GED, individuals under age 24 with little or no work experience, 
individuals who had received AFDC benefits for 36 or more of the 60 
months prior to certification, and members of households who were 
scheduled to lose their AFDC eligibility within two years because their 
dependent children would “age out” of the program. Under JOBS, 
participants in target groups had to account for at least 55 percent of all 
program expenditures. 


e Replace E&T sanctioning rules with J OBS sanctioning rules. JOBS 
rules were generally perceived as being more stringent than E&T 
sanctioning policies because they had more serious consequences for the 
second and third occurrences of noncompliance. However, under 
JOBS, sanctions applied only to the noncompliant individual, rather than 
to the entire household.°® 


To achieve conformance of E&T service designs with JOBS, states were 


encouraged to: 


e Provide assessment and individual service planning to all 
participants. In JOBS, assessment and individual service planning 
were required for all participants. Under JOBS, states could define 
assessment as a service component for the purposes of computing client 
participation hours during the first month of JOBS participation. 


e Offer the same service components available to J OBS participants 
and require clients to participate in at least 24 hours of E&T 
activities per week. JOBS service components varied by state but had 
to include education, job skills training, job readiness activities, and job 
development and placement assistance, as well as two of the following 
four optional service components: group OF individual job search, on- 
the-job training, work supplementation, and community work 
experience or another approved work experience program. Case 
management was also a permitted JOBS service component, at 
individual state option. 


e Develop nonreimbursable coordination agreements for the delivery 
of a wide range of education and training services to participants 
through individual referral arrangements. To provide more intensive 
education and training services, the JOBS legislation encouraged 
programs to develop nonfinancial coordination agreements with a range 


ET 


6 The JOBS sanctioning rules required participants to complete a 3-month and 6-month sanction 
period for second and third sanctions, respectively, before they could request to have their benefits 
reinstated. 
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of local education and training providers as a supplement to services 
provided directly with JOBS funds. 


e Use JOBS procedures to match clients to individual or sequenced 
services. In contrast to E&T service designs, which often required all 
enrollees to participate in job search or job search training as the first 
component, JOBS service assignment procedures often were based on an 
individual assessment of participant needs and participation in multiple 
services to address identified needs. 


e Require certain participants to enroll in education activities, if these 
participants had not completed high school. Under JOBS, mandatory 
work registrants who were caretakers under 24 years of age were 
required to participate in GED programs if they had not completed high 
school. Caretakers under 20 years of age had to participate in education 
regardless of the ages of their children. 


e Offer supportive services consistent with the state’s JOBS supportive 
service plan. JOBS programs usually offered a higher level of 
reimbursement for a broader range of work-related expenses compared 
to the $25 per month transportation reimbursement available under E&T 
(JOBS offered reimbursements for books, uniforms, tuition, automobile 
repair, and dental work). JOBS also offered more generous child care 
expense reimbursements. 


Each of these program features offered an alternative to the then-common E&T 
program design of universal targeting with a standardized service sequence that was 
perceived as not intensive enough to make a difference for most participants. 
Implementing uniform work registration and sanctioning procedures between E&T and 
JOBS was also attractive because it would enable states to simplify and consolidate 
time-consuming and complex administrative features of the E&T and JOBS programs. 
In addition, it was hoped that using JOBS’ tougher sanctioning procedures for 
mandatory work registrants in E&T would reduce the rate of noncompliance and free 
up staff time spent tracking participation and requesting sanctions. Staff could then 
spend more time providing employment and training services to program participants. 
States with low AFDC benefit levels—where many JOBS participants became ineligible 
for AFDC benefits as soon as they obtained a minimum wage job—also were eager to 
facilitate the transfer of individual participants from JOBS to E&T, and vice versa, 
without interrupting the delivery of employment and training services, as participants 
moved between public assistance (PA) and Food Stamps/non-PA status. 


In replicating the JOBS service model for E&T participants, states also hoped that 
they would be able to build on the coordination networks developed by JOBS to 
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leverage funds from other programs—such as the JTPA and adult education systems—to 
pay for the education and vocational training services received by program participants. 
In addition, the possibility of service consolidation for E&T and J OBS offered the 
potential to realize cost savings in the delivery of services for both JOBS and E&T 
through economies of scale. Although they were recognized to be potentially 
expensive, the enhanced supportive services available under JOBS were perceived by 


some states as key to increasing participant access to more intensive education and 


training services. 


Five states were ultimately selected for participation in the Food Stamp 
E&T/JOBS Conformance Demonstration.? The approved demonstration projects 
included: 
e Georgia’s JOBS Employment and Training (JET) Program operated 
under the umbrella of the state’s Positive Employment and Community 
Help (PEACH) Program. 

e Hawaii’s Positive Response in Developing Employment (PRIDE) 
Program. 

e Missouri’s JOBS-Employment and Training (JET) Demonstration. 

e South Dakota’s Family Independence Food Stamp Employment and 
Training/JOBS Conformance Demonstration. 


e Texas’ Better Opportunities for New Directions (BOND) Program. 


Although the official demonstration period was a full four years—from October 1, 
1992 through September 30, 1996—the participating states used the first six to 18 
months after the announcement of demonstration funding to prepare for project start-up. 
Objectives of the demonstration common to the demonstration states of Georgia, 
Hawaii, Missouri, South Dakota, and Texas included: (1) increasing administrative 
efficiency through the simplification of work registration and sanctioning procedures 
across welfare-to-work programs; (2) increasing the range and intensity of education, 
vocational training, and job search services available through the E&T program; (3) 
providing services that were well-matched to the needs of job-ready and less job-ready 


EY 


7 The official demonstration title was the E&T/JOBS/JTPA Conformance Demonstration. 
Inclusion of the Job Training Partnership Act in the official title reflected the federal government’s 
interest in improving nonfinancial coordination linkages with JTPA-funded programs to support the 
provision of individualized education and training services to demonstration participants. Strong linkages 
with the JTPA system did not emerge, however, as central features of the approved demonstrations. 
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individuals through the use of assessment and individualized service planning; and (4) 
improving client outcomes. 


Under a cooperative agreement with each demonstration state, the Department of 
Agriculture authorized the conformance demonstrations in designated project areas and 
provided each state between $500,000 and $600,000 to cover the increased costs of 
demonstration activities, including the increased costs of enhanced supportive services 
for demonstration participants. The cooperative agreements also specified that each 
state should select an independent evaluator and conduct an evaluation of the 
demonstration. 


Within the common framework established by the demonstration guidelines and 
shared state objectives, the demonstration states varied substantially in: 


e The number of local E&T sites (and percentage of all statewide work 
registrants) involved in the demonstration. 


e How priority target groups were defined, how potential participants 
were selected or recruited for services, and the extent that participation 
by volunteers was attempted and/or achieved. 


e The particular service components available to demonstration 
participants, and how participants were matched to services. 


e Whether the demonstration involved actual consolidation of E&T 
operations with JOBS or the operation of separate but parallel programs. 


The demonstration states also varied in whether they transformed their statewide 
E&T program at about the same time as, but independently of, the demonstration, or 
whether they retained the “old style” broadly targeted services in nondemonstration 
counties. To some extent, states could model their E&T programs after the JOBS 
model without receiving waivers from the Department of Agriculture. Under the E&T 
program rules in effect starting in FY 1992 (when the participation standard was 
reduced to a minimum of 10 percent), states could adopt JOBS service components, 
implement selective client targeting policies similar to the JOBS target groups, and 
consolidate the delivery of E&T and JOBS services through integrated service contracts 
or the use of integrated in-house employment and training units. In fact, three of the 
five states that were selected for participation in the E&T/JOBS Conformance 
Demonstration (Missouri, Texas, and Georgia) adopted client targeting and service 
offerings modeled after their JOBS programs for their E&T programs on a statewide 
basis. However, without an official demonstration waiver, states could not use JOBS 
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work registration or sanctioning criteria or receive federal support for the cost of 


enhanced supportive services for E&T participants. 


OVERVIEW OF THE REPORT 

This report synthesizes the findings from the states’ impact and cost evaluations. 
A companion report (Kogan and D’Amico, 1997) provides a synthesis of the states’ 
process and implementation evaluations. Both reports are based on the written 
evaluation reports prepared by the demonstration states and their independent local 
evaluators. For this report, the primary sources are the states’ impact and cost 
evaluations. However, in cases where information is not available from the impact and 
cost evaluations, we also draw on information from the process and implementation 
evaluations.’ The results presented in this report are wholly based on information 
provided by the states and their evaluators. We acknowledge their hard work in 


conducting those evaluations. 


In Chapter II, we present an overview of the demonstrations and their evaluation 
designs. We also discuss some of the methodological issues and challenges that the 


state evaluators faced and that qualify the interpretation of the results. 


In Chapter III, we discuss the findings of the states’ implementation and cost 
evaluations. In particular, we present the impacts of the demonstrations on 
participation patterns (i.e., work registration, participation rates, participant 
characteristics), on sanctioning, on services received (type and intensity), on outcomes 


(employment, earnings, and Food Stamp receipt), and on costs. 
In Chapter IV, we provide an overall summary and conclusions. 


Appendices for each state provide an overview of the state’s demonstration, 
describe its evaluation design, and summarize its findings. These appendices do not 
describe all the analyses conducted by the states’ evaluators. Instead, the appendices 
present summaries of the findings relevant to the topics discussed in this synthesis. 


na 


8 A list of references at the end of this report lists the state evaluation reports on which this 


synthesis report is based. 
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If. OVERVIEW OF DEMONSTRATION EVALUATION DESIGNS 


SUMMARY OF THE DEMONSTRATIONS’ DESIGN AND IMPLEMENTATION 


Although each demonstration was unique, they shared a number of common 
features. As summarized in Table II-1, a majority of the five demonstrations: 


e Replaced E&T work registration requirements with JOBS work 
registration requirements. 


e Encouraged volunteers. 
e Used targeting criteria similar to JOBS. 
e Implemented more stringent sanctioning rules. 


e Called in only a subset of work registrants for services to address 
service capacity limitations. 


e Provided increased assessment and individualized service planning. 


e Increased access to education and training through either E&T-funded 
services or referrals to other providers. 


e Provided increased access to supportive services. 


e Did not consolidate E&T and JOBS operations, but operated similar 
parallel programs. 


Thus, the demonstrations typically provided more intensive services to a 
somewhat more limited population, with a focus on serving volunteers and priority 
groups. This more intensive, but narrower, approach was accompanied by more 
stringent sanctioning of those individuals who failed to meet E&T participation 


requirements. 


This broad similarity among the demonstrations enables us to synthesize the 
results of the individual demonstrations to come to some overall conclusions regarding 
the impacts of this general package of program design changes. The analyses 
conducted by the states, however, do not usually allow us to assess the impacts of 
individual features of the demonstrations, such as more stringent sanctioning rules, 
although some of the analyses are suggestive. Rather, each the analysis addresses the 
impacts of the state’s demonstration as a whole, in comparison to an estimate of what 
would have occurred in the absence of the demonstration. 
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Table II-1 
Key Features of Demonstrations 


SS Se 


Scope of demonstration 
Number of counties 4 1 8 20 1 


Percentage of E&T work registrants who 16% 70% 16% 100% 2% 
are in demonstration counties 


Participation and sanctioning 


Fewer exemptions from work registration ¥ va A ve Me 

Volunteers encouraged v v a ve 

Selective targeting similar to JOBS v Me v v 

More stringent sanctioning A vi A sh 

Calls-in restricted by service capacity v “a va 

Service designs 

Increased assessment/service planning v, v v ve 

Case management v v Se 

Increased education and training v v vi wi v 

Increased supportive services a v v 

Consolidation with JOBS 

Full consolidation A 

Consolidated with separate case managers v 

Little consolidation ve yi v 

Changes to nondemonstration sites 

None v n/a 

Similar changes to service design, but not v v n/a v 

as substantial 

Comparison strategy 

Pre/post comparison f 

Comparison site v v 

Combined pre/post, comparison site % v 
v 


Within site random assignment (partial) 
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Although the demonstrations were broadly similar, in interpreting the results of 
the state evaluations we must keep in mind that there are some substantial differences 
between the demonstrations. Below, we briefly discuss the key features of each state’s 
demonstration design, based on a synthesis of the states’ implementation and process 
evaluations (Kogan and D’Amico, 1997), which examined the implementation of the 
demonstrations in considerable detail. 


KEY FEATURES OF STATE DEMONSTRATION DESIGNS 


Georgia. Georgia’s JET demonstration emphasized: (1) participant access to a 
more flexible sequence of component services, in combination with individualized 
assessment and one-on-one case management for all clients, and (2) more generous 
supportive services, including enhanced assistance for training-related expenses and 
post-employment services. In combination, these demonstration features resulted in 
(3) more frequent utilization of education and training activities by participants, 
including participation in self-initiated training by a number of volunteers. 


Hawaii. Among the key program features tested by the PRIDE demonstration 
were: (1) an effort to identify a wide range of employment barriers faced by 
participants and link individuals to community agencies that can help address those 
barriers; (2) the design and delivery of preemployment services to address their 
sometimes low motivation and self-esteem as well as to provide participants with 
improved job seeking skills; (3) an emphasis on participation by work registrants from 
priority target groups that can benefit from more intensive E&T services and by both 
mandatory and exempt work registrants who volunteered for services; and (4) an 
emphasis on encouraging participation in education and training services, particularly 
for participants who have not completed high school. 


Missouri. Key elements of the JET service design included: (1) a strong 
emphasis on serving volunteers, (2) an emphasis on comprehensive assessment, careful 
service planning, and ongoing case management for each participant, and (3) access to a 
wider range of employment and training services than was typical for E&T participants 
prior to the demonstration or was typical in nondemonstration counties. As a 
consequence of these attributes, JET served relatively few participants and a much 
smaller proportion of the demonstration area’s mandatory work registrants than was 


customary in the nondemonstration counties. 
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South Dakota. The key program features tested by the South Dakota Food 
Stamp E&T/JOBS conformance demonstration were: (1) an expanded and 
individualized service planning process; (2) an increased emphasis on utilization of 
education services, particularly for individuals without high school completion; (3) little 
encouragement given to voluntary participation; and (4) a continued interest in calling 
in a high percentage of all mandatory work registrants. The demonstration model was 


used statewide for all counties serving E&T participants. 


Texas. The BOND demonstration was characterized by: (1) full consolidation of 
E&T and JOBS operations; (2) a high level of participation by exempt volunteers, (3) 
the development of a two-tier service design offering an increase in the intensity of job 
readiness and job search services for more job ready participants and access to basic 
and postsecondary education and training for individuals with more serious employment 
barriers; and (4) a significant investment in child care expenditures to support 


attendance in long-term education and training. 


EVALUATION DESIGNS 

The key evaluation design decision faced by each state was choosing a 
comparison method. To conduct a valid impact evaluation, states needed to develop a 
method of estimating what would have happened in the absence of the demonstration. 
In this way, the impact of the demonstration could be derived by comparing outcomes 
achieved in the demonstration to what outcomes would have been had there been no 
demonstration. There are three basic comparison strategies that states could have used. 


e Pre/post. States could compare outcomes during the demonstration with 
outcomes in the same site before implementation of the demonstration. 


e Comparison site. States could compare outcomes in the demonstration 
site to outcomes in other, nondemonstration sites. 


e Within-site random assignment. States could randomly assign 
individuals to receive either demonstration or nondemonstration services 
and then compare the outcomes for these two different groups of 


participants. 
Each approach has its own advantages and disadvantages, none is clearly superior to the 
others in all respects. 
Comparison Sites 


Two states, Georgia and Missouri, relied on a comparison site design to evaluate 


their conformance demonstrations. The validity of this strategy relies on the 
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assumption that the introduction of the demonstration is the only difference between the 
demonstration and comparison sites that has substantial effects on the outcomes being 
examined. To help assure that the demonstration and comparison sites were similar, 
Georgia selected a matched comparison county for each of the counties included in the 
demonstration. Paired counties were generally similar in terms of the size of the Food 
Stamp Program; the ethnicity, age, and gender of Food Stamp Program participants; 
the number of work registrants; and the local unemployment rate. Missouri also had 
planned to select matched comparison sites, but was unable to implement this plan. 


Clearly, it is impossible to select comparison sites that are identical to the 
demonstration site in all relevant respects. For example, comparison sites will 
generally serve a clientele with somewhat different demographic characteristics. For 
this reason, the state evaluators chose to use multiple regression for some of their 
analyses. This multivariate technique enables them to control for observed differences 
in participant characteristics between the demonstration and comparison sites. While 
this approach is effective, it does not address differences that are not measured. For 
example, the demonstration sites typically encouraged volunteers and, as a result, a 
greater percentage of participants in the demonstration sites were volunteers. 
Volunteers are likely to be more motivated and differ from nonvolunteers in other 
unmeasured, but important respects. Some states, however, (e.g., Georgia) were able 
to control for volunteer status in some regression analyses. 


The ability of some states to control for differences in participant demographics 
was also impaired by data limitations, especially in the comparison sites. For example, 
Georgia was able to include only a few characteristics in its regression models 
comparing the demonstration and comparison sites. In addition, it can be difficult to 
control for changes in local economic conditions that affect the comparison and 
demonstration sites differentially because measures of economic conditions are often 
highly correlated with the variable identifying the demonstration site. 


An advantage of this comparison-site approach is that both the demonstration and 
comparison sites would be subject to changes over time that affect the entire E&T 
system, such as the reduction in the federal participation rate standard or changes in 
economic conditions that are uniform throughout the state. In both Georgia and to a 
lesser degree in Missouri, there was a shift toward individualized service planning and 
a broadening of referral and services offered throughout the state. Instead of 
concentrating solely on job search, participants also could be referred to other training 
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programs such as postsecondary, vocational or on-the-job training. Asa result the 
comparison sites became more similar to the demonstration sites. Hence, we can 
expect the measured impacts of the demonstration to be smaller than they would have 
been if this change had not affected the comparison site. This does not mean that the 
estimates are invalid. It does however, affect their interpretation because the 


counterfactual has changed. 


Within-Site Random Assignment 

Georgia also implemented a within-site, random assignment design to determine 
the effects of increased supportive services. This approach has substantial statistical 
advantages over the alternatives because the random assignment itself assures that the 
demonstration and comparison samples are statistically equivalent. This approach was 
not generally used for the demonstration evaluations because the demonstrations were 
examining the effects of systemic changes that were difficult or impossible to limit to a 
random subset of participants. For example, many operational rules and procedures 
were changed by the demonstration. To implement them only for a subset of 
participants would require E&T program staff to implement different operational 


procedures for different participants, a difficult task. Further, even if they succeeded in 


taking this approach, the dual approach would forgo the cost savings expected from 
simpler administrative procedures. Georgia’s approach, however, enables them to 
draw conclusions about the influence of increased supportive services apart from the 


influence of the rest of the demonstration. 


Pre/Post Comparison 


Only one state, South Dakota, chose to rely solely on a pre/post comparison for 
its evaluation. Pre/post comparisons risk confounding the estimated effects of the 


demonstration with the effects of other changes that occur during the evaluation period. 


First, additional changes in the E&T program not formally part of the demonstration 
could influence the change in outcomes. For example, in the transition from FY 1992 
to FY 1993, changes may have occurred in the statewide E&T program design to 
prepare for the planned change to outcome-based standards or in response to the 
reduction of the federal participation rate standard from 50 percent to 10 percent. 
Thus, services and outcomes might have changed even in the absence of the 
conformance demonstration. Second, changes in general economic conditions may 
have substantial effects on participant outcomes. Finally, the characteristics of 
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participants might change over time, in response to either the demonstration itself or to 
other changes in the state environment. 


To address the threats to the validity of the evaluation posed by these potential 
confounding factors, South Dakota’s evaluation employed multivariate models that 
controlled for differences in participant characteristics and for temporal and geographic 
differences in economic conditions. Participant characteristics used in these regression 
models included, for example, gender, race, and age. Local economic conditions 
included the number of employed and unemployed in the county. These multiple 
regression models were used to examine two outcomes, employment and earnings; 


other analyses relied on simple pre/post comparisons. 


Combined Pre/Post and Comparison Site 

Two states, Texas and Hawaii, used a combined pre/post, comparison site 
strategy for estimating the effects of the demonstration. The impact of the 
demonstration is measured by the difference between the demonstration and comparison 
sites of the change from before to during the demonstration period. For example, if 
earnings increases by $15 in the demonstration site and increases by $5 in the 
comparison site, then the estimate of the impact is $10 (15 - 5). Both states, however, 
were unable to use the full design for all outcomes because some outcomes were not 
available in the predemonstration period. For these outcomes they used a comparison 


site design. 


The strength of the combined design is that it adjusts both for preexisting 
differences between the demonstration and comparison sites and for trends over time 
that are common to both sites. Thus, the main threat to the validity of the estimates is 
differential changes in participant characteristics or economic conditions. To address 
this possibility, both states’ impact evaluations estimated regression models that 
controlled for a variety of participant demographic characteristics and economic 
variables. Because there were only one demonstration and one comparison county in 
each state, the county-level economic variables used in the regressions are likely to be 
highly correlated with the variable measuring the demonstration impact. This 
correlation may have seriously reduced the statistical power of the regression models. 


In Texas, the E&T program in the comparison site experienced some changes that 
made it more like the demonstration. The most important of these changes was a 
statewide redesign of the job search and job readiness components offered to E&T 
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participants that occurred just as the demonstration was beginning. As a result of this 
consolidation, E&T participants received expanded and intensified job search and job 
readiness services, which were similar in both the demonstration and comparison sites. 
In addition, transportation allowances available to E&T participants were increased. 
As a result of these changes in the comparison site, the major service-design features 
that distinguished the demonstration site from the comparison site were (1) increased 
access to a wide variety of additional activities, including initial assessment, group or 
individual case management, education, vocational skills training and work experience 
and (2) increased access to supportive services, especially child care assistance. The 
estimated impacts of the demonstration should be interpreted as those resulting from 


these remaining differences. 


Hawaii implemented “work first” requirements in the JOBS program beginning in 
April 1995. These requirements, which were introduced into the demonstration as 
well, required all participants, including those in education and training components, to 
work at least eight hours a week. In addition, a dramatic reduction in state funding led 
to a redesign of the demonstration service delivery design in July 1995—emphasis 
shifted from barrier removal and employability development to immediate employment. 
Before July 1995, however, the E&T program in the comparison site operated without 
major organizational or service redesigns. To avoid the effects of these changes, the 
impact evaluation covered the period from January 1994 to June 1995. Although most 
of the evaluation period occurs before the program changes, there is some evidence that 
the work first requirements may have induced demonstration participants to drop out of 
training and education activities near the end of the evaluation period. The 
demonstration findings, therefore, should be interpreted with this qualification in mind. 


Evaluation Samples 

One other methodological issue affected all the state evaluations. All states 
included only E&T “participants” in their evaluation samples for many analyses; only a 
few analyses were based on all work registrants. Although the definition of participant 
varied among states, in all cases it was only a small subset of work registrants. The 
demonstrations, however, were expected to affect who participated because they 
encouraged volunteers, implemented selective targeting, and generally served fewer 
individuals so that more intensive services could be provided. Two problems result for 
the demonstrations’ estimated impacts. First, the impact of the demonstration is not 
limited just to those individuals who participate; some nonparticipants are affected 
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because they did not receive services that they would have received without the 
demonstration. Thus, even if outcomes are increased for participants, that increase 
might be offset by a reduction in outcomes for nonparticipants. The state evaluations 
ignore this possible effect and, thus, tend to overstate the overall impact of the 
demonstration. Second, we can expect the demonstration to affect the characteristics of 
participants so that participants in the demonstration and comparison sites are 
necessarily not comparable. To control for demonstration-induced differences in the 
characteristics of participants, most state evaluators used regression analysis to adjust 
for measured difference in participant characteristics, as discussed above. 


SUMMARY 


Overall, the states developed evaluation designs that were intended to overcome 
the major challenges to developing valid estimates of demonstration impacts. Their 
ability to develop reliable estimates was, however, compromised by several factors. 
First, the systemic nature of the demonstration precluded the states’ evaluators from 
using the most reliable evaluation design, within-site random assignment. Thus, states 
were forced to use the alternatives of comparison site and pre/post designs. Second, 
redesigns of the comparison programs or of the demonstration itself midway through 
the testing period affected the interpretation and meaning of the results in several states. 
Third, the ability of the evaluators to control for confounding factors was limited by 
data availability, especially in the comparison sites, and by strong correlations between 
economic variables and the variables used to identify the demonstration impact. 
Finally, data in some states were not available for some of the outcomes of interest in 
the evaluation. Overall, although some individual state evaluations are inconclusive or 
incomplete, the combined efforts of the evaluators provide us with sufficient 
information to draw conclusions about the effects of the demonstrations as a whole, at 
least for many of the dependent variables examined. In the next section, we summarize 
the overall findings of the states’ evaluations. 
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Il. ESTIMATED IMPACTS OF THE DEMONSTRATIONS 


The demonstrations were intended to transform E&T systems to conform to the 
JOBS program in order to: 


e Streamline administrative procedures to reduce duplication of effort and 
save on administrative costs. 


e Increase compliance by imposing more stringent sanctioning procedures. 


e Selectively target resources to those most likely to benefit and to 
volunteers. 


e Individualize service plans to match the intensity of services to 
participant needs. 


The ultimate objective was to help improve the labor market outcomes of participants 
and thereby reduce Food Stamp receipt by individuals and Food Stamp-related costs to 
taxpayers. In this section we summarize the findings of the state impact and cost 
evaluations. We discuss the effects on participation patterns, on the types and intensity 
of services received, on labor market outcomes, and on costs. 


PARTICIPATION PATTERNS 


Four aspects of the demonstrations can be expected to affect participation 
patterns: the more inclusive work registration requirements, selective targeting of 
resources toward priority groups, encouragement of participation by volunteers, and 
tougher sanctioning procedures. These changes can be expected to affect work 
registration, participation rates, service to volunteers, the characteristics of participants, 
and the frequency with which individuals are sanctioned for noncompliance. 


Mandatory Work Registrants 


All five demonstration states implemented JOBS work registration rules as part of 
their conformance demonstrations.! The shift to JOBS rules expanded the pool of work 
registrants in most states to include UI claimants and caretakers of children between the 


1 Both E&T and JOBS participation rules exempt individuals under 18 or over 60 years of age as 
well as individuals needed in the home to care for dependent children. The E&T exemption rules are 
more lenient than were the JOBS rules in several respects, however. First, E&T exempts all caretakers 
of children under six years of age, while JOBS exempted only caretakers of children under three years of 
age (or one year of age at state option). Second, E&T participation rules also exempt individuals enrolled 
in school at least half-time and UI claimants in compliance with UI work requirements. 
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ages of three and six. These more inclusive work registration requirements were 
expected to increase the number of work registrants in the demonstrations. As shown 
in Table III-1, there were substantial increases in work registration in Hawaii, South 
Dakota, and Texas and a modest increase in Georgia; there was virtually no change in 


Missouri. 


Table III-1 
Estimated Impacts on Work Registration 


South 
Georgia Hawaii Missouri Dakota Texas 


EE ES eS 


Change in number of mandatory +9% > 60% None +49% +25% 
work registrants (%) 


Note: Data are not comparable among states. The Texas estimate is based on a simulation of changes in 
exemptions. 


Most of the increase in work registration appears to result from reducing the 
exemption for caretakers of young children. The Texas evaluation estimated that this 
group accounted for about 80 percent of additional registrants under the demonstration 
work registration rules. The South Dakota evaluation found an increase in the 
percentage of work registrants who were female from 44 percent to 57 percent and an 
increase in the percentage who were parents with children from 48 percent to 63 


percent. 


Participation Rates 

The demonstrations were intended to both target services to priority groups and to 
provide more intensive services. As a result the number of participants (i.e., 
individuals who received at least one reportable E&T program service component) was 
expected to decline as resources were devoted to serving fewer people with more 
intensive services drawn from a larger work registrant population. To examine this 
issue, we look at participation rates, defined as the percentage of work registrants who 


received E&T component services. 


As shown in Table III-2, participation rates indeed fell in most states. In this and 
other tables, the data are not always comparable among states due to differences in 
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definitions. The reader should focus, therefore, on the direction and relative size of the 
state differences. The largest reductions in participation rates under the demonstration 
occurred in Hawaii and South Dakota (which experienced the largest increases in the 
size of their work registrant pools) and Missouri (which sharply reduced the number of 
demonstration E&T participants compared to the previous E&T program). There was 
only a modest reduction in participation rates in Georgia. In Texas, the participation 
rate may have increased somewhat due to the demonstration—although the participation 
rate fell in the demonstration site, it fell by a greater amount in the comparison site. 
After these changes, there were still substantial differences among states in the 
participation rate. Missouri and Texas had very low participation rates (1 percent and 3 
percent, respectively), while the other states continued to provide E&T services to at 


least 15 percent of work registrants. 


In all states, the decline in participation rates more than offset the increase in 
work registration. Thus, the number of participants decreased during the 


demonstrations. 
Table III-2 
Estimated Impacts on Participation Rates 
South 
Georgia Hawaii Missouri Dakota’ Texas 

Comparison participation rate? 20% 88% 21% 31% 1% 
Demonstration participation rate 15% 32% 1% 15% 3% 
Change in participation rate -5 -56 -20 -16 ae! 


(percentage points) 


2 In this and subsequent tables, the comparison data represents the comparison information used by 
the state’s evaluation and can be one of several alternatives: actual data in the comparison site during the 
demonstration period, actual data in the demonstration site from before the demonstration period, or a 
calculated comparison based either on the pre/post difference in change analysis or on an estimated 
regression model. The demonstration data are actual values from the demonstration site during the 


demonstration period. 
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Note: Data are not comparable among states. Participation rates are calculated based on all mandatory 
work registrants for Texas, Missouri, and South Dakota; on work registrants and volunteers called in for 
service in Georgia, and on mandatory work registrants who were called in for Hawaii. For Texas, the 
calculated comparison is derived from a regression model. In Georgia, individuals attending the first 
face-to face meeting with a case manager are counted as participants even if they receive no subsequent 
services. In Texas and Hawaii individuals are not counted as participants unless they receive at least one 
hour of an activity beyond assessment. Participation is not defined consistently in the Hawaii 


demonstration and comparison sites. 


Volunteers 

Most of the conformance demonstrations were designed to encourage 
participation by volunteers who were not mandatory work registrants. The only 
exception was South Dakota, which did not serve volunteers. Three of the other states, 
Georgia, Missouri, and Texas, served substantial numbers of volunteers, as shown in 
Table III-3. The Hawaii demonstration served only a small percentage of exempt 
volunteers. However, nearly 60 percent of mandatory work registrants were classified 


as volunteers because voluntarily participated before being called in. 


Two states, Georgia and Texas, examined the characteristics of volunteers. Both 
states found that volunteers were more likely to be female and were more highly 


educated than others. In Georgia, volunteers also tended to be younger, while in Texas 


they tended to be white. 


Table III-3 
Estimated Impacts on Service to Volunteers, Otherwise Exempt 
(Percentage of Participants) 


South 
Georgia Hawaii Missouri Dakota Texas 
Comparison (%) 0% 32% 0% 0% 0% 
Demonstration (%) 28% 4% 56% 0% 40% 
+14 -28 +56 0 +40 


Change (percentage points) 


Note: Data are not comparable among states. Volunteers from among bo 
registrants are counted in Texas. 


th exempt and mandatory work 
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Priority Groups and Participant Characteristics 

All the state demonstrations, except in Texas, designated high-risk target groups 
that were to be given priority for service in the demonstration E&T program. Often 
these priority groups were modeled on those used by JOBS. The most common priority 
groups included: 


e Under age 24 without a high school diploma or equivalent (GA, HI, 
MO, SD). 


e Under age 24 with little or no recent work experience (GA, MO, SD). 
e Little or no work experience (GA, HI, MO). 

e Long-term food stamps receipt (GA, HI, MO). 

e No high school diploma or equivalent (GA, MO). 


Overall, the state evaluations tended to indicate that the demonstrations were 
successful in focusing services among these priority groups, as measured by the 
percentage of participants who were priority group members. For example, in Georgia 
all but the lowest priority group had higher participation rates than those who were not 
members of any priority group. In Hawaii, over 70 percent of participants were 


members of at least one priority group. 


Although most states focused services on priority groups, the sharp decline in 
participation rates in some states may have caused the number of individuals served in 
some of these groups to decline. For example, the participation rate in Missouri fell 
from 21 percent to 1 percent so that the numbers served in virtually any priority group 
are likely to have declined. 


As states focused their E&T services on these groups one would expect that the 
percentage of participants who were young, were high school dropouts, and had little 
work experience would increase. There were, however, some other forces affecting the 
characteristics of participants. The most important was the encouragement of 
volunteers. Because volunteers tended to be more educated than mandatory 
participants, as discussed above, encouraging service to volunteers led to increased 
service to more educated individuals and other groups interested in receiving the 
services offered by the demonstrations. Therefore, service to volunteers worked 
against the goal of serving those most in need and tended to offset some of the effects of 
selective targeting. In addition the change in work registration requirements, especially 
the elimination of the exemption for caretakers of young children, could be expected to 
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increase the percentage of participants who were female and the percentage that were 


young. 


Three of the state impact evaluations examined the characteristics of participants. 
Table IlI-4 summarizes the available evidence for characteristics examined by several 
states. The results show that, as expected, the demonstrations led to increases in the 
percentage of participants who were female, were young, or had limited work 
experience. Somewhat surprisingly, the percentage of high school dropouts among 
participants decreased in the two demonstrations that provided information. 


Table III-4 
Estimated Impacts on Participant Characteristics 
(Percentage Points) 


South 

Missouri Dakota Texas 
Female +29 +20 +29 
White +30 ue) +23 
High school dropout -4 N/A -26 
Age under 25 (SD) or 30 (TX) N/A +11 +27 
Limited work experience +32 N/A N/A 
Long-term food stamps recipient N/A +4 N/A 


The major factor behind this decrease is likely to be the encouragement of volunteers— 
as discussed above, volunteers tended to be better educated. Three of the states found 
that the demonstrations increased the percentage of participants who were white. Other 
analyses conducted by the states tended to show that the demonstration tended to 
increase participation by adults with children. Overall, the major factors affecting 
participant characteristics appear to be the elimination of the exemption for caretakers 
of young children, which led to increased relative service to young women, and 
encouraging volunteers, which may have led to increased service to high school 


graduates interested in further education. 
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Sanctioning 


Most of the demonstrations implemented the tougher sanctioning rules used by 
JOBS. The two exceptions were Hawaii and Missouri. In Hawaii, the conformance to 
JOBS sanctioning procedures weakened existing sanctioning procedures because of the 
lenient conciliation procedures used by JOBS until JOBS conciliation procedures were 
modified in the Spring of 1995. In Missouri, the revised sanctioning procedures were 
not implemented until the spring of 1996 due to an administrative oversight. 


The expected effects of stricter sanctioning roles are ambiguous—they can lead to 
either increased sanctioning or greater compliance. As shown in Table III-5, there 
were substantial declines in the percentage sanctioned in Georgia and in requests for 
sanctioning in Texas. The modest changes in Hawaii may have led to a slight reduction 
in the percentage of work registrants sanctioned (notices of adverse action). 


Table ITI-5 
Estimated Impacts on Sanctions 


South 
Georgia Hawaii Dakota Texas 
Comparison sanctioning rate 36% 6% 49% 60% 
Demonstration sanctioning rate 21% 5% 46% 43% 
Change (percentage points) -15 -] -3 -17 


Note: Data are not comparable among states. Sanctioning rates are based on selected work 
registrants in Georgia, on all work registrants in Hawaii, and on requests for sanctions and 
the number of called-in work registrants in Texas. Missouri did not implement changes in 
sanction procedures. 


There was, however, an increase in the number of sanctions because the number of 
mandatory work registrants increased substantially. In South Dakota the estimates 
show a slight decrease in sanctions. Subsequent data indicate, however, that there may 
have been an increase during the later part of the demonstration. Overall, the 
demonstrations appear, however, to have reduced sanctions. Data from Georgia 
suggest that the lower sanction rates may have resulted in part from increased 


compliance. 
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E&T SERVICES 

All of the state demonstrations were intended to increase the range and intensity 
of education and training services offered. Thus, states offered more educational and 
job training activities in addition to the job search or job readiness training components 
that they had concentrated on before the demonstrations. States also tended to increase 
service planning and case management so that services would be more carefully 
matched to the individual participant’s needs and interests. Below we examine the 
impacts of the demonstration on the types of services received by participants, on the 
intensity of services, and on the completion of services. 


Types of Services 

All state demonstrations offered a wider range of services to participants; 
expanded services offered by many of the demonstrations included education (e.g., 
postsecondary, GED, ABE, etc.), vocational training, and work experience. States, 
however, differed substantially in the extent to which participants received services 
beyond the traditional job search and job readiness services previously emphasized by 


their E&T programs, as shown in Table III-6. 


In virtually all states, the provision of job search activities dropped substantially. 
The largest reductions occurred in Georgia where individual job search declined by 34 
percentage points and job search skills training by 34 percentage points. But most other 
states also had substantial reductions. The only exception was South Dakota where 
there was only a small drop in the provision of these services. Nonetheless, except for 
Texas, states continued to provide job search related services to at least 40 percent of 


participants. 


All states increased provision of enhanced services such as education services and 
vocational training. Texas expanded services beyond job search and job search training 
by the largest percentage of participants, about 70 percent. South Dakota provided 
such enhanced services to only an additional quarter of participants. The other states 
provided enhanced services to about an additional 40 percent of participants. 
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Table ITI-6 
Estimated Impacts on Receipt of EXT Services 
(Percentage Points) 


South 

Georgia Hawaii Missouri Dakota Texas 
Individual job search, directed -34 -40 -20 ~4 -66 
job search, job entry 
Job search skills training, job -80 ota -18 -9 -3 
readiness, group job search 
Education services ap ere -5 al +14 +65 
Vocational training sas sew As) -6 ae tat 
Work experience, community a) aly -3 +4 +4 
service 
On-the-job training N/A N/A -2 te N/A 


Note: Data are not comparable among states. Categories have been combined both across and within 
states. N/A indicates that the state did not use the category or did not provide information for the 
category. Missouri data are based on the total number of services, not the number of participants, and 
therefore the percentage point changes sum to zero after adding a 22 percentage point increase in 
assessment. Effects in Missouri for participants would be larger in magnitude because some participants 
received multiple services. 


States also differed in which enhanced services were emphasized. Educational 
services were most important in Georgia, Missouri, South Dakota, and Texas. Three 
of these states (Georgia, Missouri, and Texas) also served a high percentage of 
volunteers; their high provision of educational services may have occurred because 
volunteers wanted support for previously planned educational activities or took 
advantage of generous supportive services to begin education or training courses. For 
example, in Georgia two-thirds of participants in postsecondary education were 
volunteers. Missouri provided a fair amount of vocational training in addition to 


educational services. Hawaii emphasized vocational training and work experience. 


Overall, the demonstrations were successful in shifting services away from job 
search and towards education services or vocational training. Only South Dakota, 
where there were few volunteers, continued to provide the vast majority of participants 


with only job search services. 
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Intensity of Services 

Most of the demonstrations also intended to increase the intensity of services. 
However, only three states examined service intensity in their impact evaluations. The 
results were mixed. Hawaii examined the duration of service for persons who 
completed a specific service and found that the length of basic education decreased by 
1.6 months while the length of job search skills training increased by 1 month. We do 
not, however, know the effect of the demonstration on the overall duration of services, 
because the overall effect also depends on the mix of services and the duration of 


services for those who do not complete. 


For South Dakota, the time spent in job readiness and job search programs was 
not affected by the demonstration. Because the enhanced service components tended to 
be longer (about 3 months as compared to 1.2 months for job readiness and 2.4 months 


for job search), we can be confident that the overall length of services increased. 


Texas found that the overall monthly hours in activities increased by nearly 50 
percent relative to the comparison site, an increase of 26 hours per month, primarily 
due to the large amount of vocational and educational training provided. 


Overall, it is likely that the state demonstrations increased the intensity of services 
because they tended to shift away from typically less intensive job search and job 
readiness activities towards typically more intensive educational and vocational training 
activities. However, we have little information about the size of the effect. 


LABOR MARKET OUTCOMES 

The emphasis of the demonstrations on providing a more intensive set of services 
to a more selected group of participants was intended, in part, to lead to improved labor 
market outcomes for participants. By providing skill-enhancing services to 
participants, it was expected that they would become more likely to be employed and 


obtain higher earnings. 


Each of the state evaluations examined the impacts on employment and earnings. 
Some states examined outcomes at exit from the program, others examined outcomes 


about a year later as shown in Table Ill-7. 
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Table III-7 
Estimated Impacts on Employment and Earnings 


South 
Georgia Hawaii Missouri Dakota Texas 
Employment (percentage points) ani, 0 2h = 0 
Average earnings (both +$63 N/A +$100 ? N/A 
employed and not employed) per qtr. per week 
Average earnings -$208 0 N/A N/A 0 
(among those employed) per qtr 


Note: Data are not comparable among states. Data for Georgia are based on all called-in work 
registrants, not just participants, 9 to 12 months after program completion. Data for Hawaii and Missouri 
represent outcomes for participants at program completion. Data for Texas represent outcomes both at 
program completion and one-year later, for which the results were similar. South Dakota examined 
outcomes only for individuals receiving job search or job readiness training, which comprised the bulk of 
all participants. N/A indicates that the state did not provide information for the outcome. 


Three states, Georgia, Missouri, and South Dakota, found that the demonstration 
increased employment. Because these analyses did not adjust for demographic or 
economic differences between the demonstration and the comparison, they are not 
conclusive. Indeed, the remaining two states, Hawaii and Texas, found no significant 
employment effect after controlling for participant characteristics and economic 
conditions, even though there were positive effects in simple difference in means 


analyses. 


Georgia and Missouri also provided information on impacts on average earnings 
(of both those employed and those not employed), that increased, primarily because of 
the increase in employment. The results regarding earnings for South Dakota were 
mixed, but tended to suggest reductions in earnings due to the demonstration. 


The results for earnings of those employed were more mixed. Georgia found a 
negative effect. Texas and Hawaii found no significant effect, but these analyses lacked 


statistical power. 


Overall, the demonstrations may have led to increases in employment and 
earnings among participants, although the effects are modest and the evidence is not 
conclusive. And these modest effects might be offset by negative effects on 
nonparticipants who would have received services under the state’s normal E&T 
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program. The only state with large estimated impacts was Missouri, which experienced 
a 21 percentage point increase in employment and a $100 per week increase in 
earnings. It is important to note, however, that these impacts were estimated for 
participants. Because the participation rate dropped dramatically in Missouri (from 21 
percent to 1 percent), it is quite possible that there are offsetting declines in outcomes 
for those work registrants who would have received services in the absence of the 
demonstration, but did not. Further, the Missouri estimates did not adjust for 
differences in economic conditions or for some relevant demographic characteristics or 
for service to motivated volunteers who might have found jobs without the 


demonstration. 


Foop STAMP BENEFIT RECEIPT 
A primary goal of the demonstrations was to reduce Food Stamp receipt by 

moving participants to self-sufficiency. Only two states examined the effects of the 
demonstrations on Food Stamp receipt, with mixed and somewhat perplexing results. 
Although Georgia found that Food Stamp receipt decreased among demonstration 
participants at exit from the E&T program, a year later Food Stamp receipt among all 
work registrants was actually higher in the demonstration sites. The South Dakota 
evaluation found increases in Food Stamp receipt during the first year after referral to 
E&T, but no difference two years after referral. Overall, the results on Food Stamp 
receipt must be considered inconclusive. There is no strong evidence that Food Stamp 


receipt was reduced by the demonstrations. 


CosTs 

All demonstration states increased funding for the demonstration site to partially 
finance the enhanced E&T and support services offered by the demonstration. Table 
III-8 examines the change in total costs (both federal and state funds), including the 
costs of support services (leveraged funds are not included). As shown in the table, 
increases in aggregate cost ranged from 43 percent in South Dakota to 368 percent in 
Hawaii. The relatively low percentage increase in South Dakota probably occurred 
because South Dakota operated the demonstration wherever the E&T program was 
operational in the state. The other states operated the demonstration in only a relatively 
few counties and could increase expenditures for the demonstration without a 


commensurate percentage increase in the state’s entire E&T budget. 
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Table III-8 
Aggregate Costs During and Before the Demonstration 


(Annualized) 
South 
Georgia Hawaii Missouri Dakota Texas 
Comparison (before) $251,193 $194,672 N/A $488,463 $250,695 
Demonstration $537,295 $911,528 $318,589 $699,947 $919,664 
Change ($) $286,102 $716,856 N/A $211,211 $668,696 
Change (%) 113% 368% N/A 43% 267 % 


Note: Data are not comparable among states. Comparison data are from the demonstration site before 
the demonstration, except in Georgia, where comparison data are from the comparison site during the 
demonstration. Comparison data are not shown for Missouri because data from before the demonstration 
were not provided and the comparison site is much larger than the demonstration site. Aggregate amounts 
are not comparable across states because of the wide variation in the size of the demonstration sites. 


Because the demonstrations experienced reduced participation rates, the cost per 
participant increased by even greater amounts than aggregate costs. As shown in Table 
IlI-9, the largest increase occurred in Hawaii, which increased costs per participant by 
a factor of 11. Costs per participant quadrupled in Georgia, Missouri, and Texas while 
they increased by “only” 121 percent in South Dakota. The relatively small increase in 
South Dakota is not surprising because, as discussed above, it experienced the smallest 
increase in aggregate costs and the smallest change in overall service mix. The 
extremely large increase in Hawaii results from the large increase in total costs 
combined with a substantial reduction in the participation rate. While costs per 
participant ranged from $141 to $458 before the demonstration, they ranged from $311 
to $1,733 during the demonstration. 
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Table III-9 
Cost per Participant During and Before the Demonstration 


South 
Georgia Hawaii Missouri Dakota Texas” 
Comparison (before) $227 $161 $317 $141 $296 
Demonstration $909 Si a733 $1,274 S31 POV Sii91 
Change ($) $682 SIY2 $957 $170 $895 
Change (%) 300% 976% 302% 121 Ge? 302% 


Note: Data are not comparable among states. Comparison data are from the demonstration site before 
the demonstration except in Georgia and Missouri, where the comparison data are from the comparison 
site during the demonstration. Costs per participant in South Dakota are based on individuals assessed; 
many individuals were assessed but did not receive subsequent services. 


Table III-10 displays the percentage distribution of costs among 3 major cost 
categories: administration, direct delivery, and support services. Support services 
include travel allowances, child care assistance, and other similar costs. Direct 
delivery includes the costs of assessment, case management, training, and other 


Table III-10 
Percentage Distribution of Costs 


South 
Georgia Hawaii Missouri Dakota Texas 


a a eee 


Administration N/A NIA 20% 6% N/A NA 15% 15% 6% 8% 


Direct delivery, 42% 80% 13% 715% 61% 77% 81% 719% 43% 83% 
training, tuition 


reimbursement 
Support services 58 % 20% 2% 19% 39% 23% 4% 6% SI% 8% 
Total 100% 100% 100% 100% 100% 100% 100% 100% 100% 100% 


Child caree 45001. 3 Win Aaa Ace ANA NIA. UNA. | 44% N/A 


Note: D represents demonstration site. C represents comparison. Comparison data are from the 
comparison site, except in South Dakota, where the comparison data are from the demonstration site before 
the demonstration. Direct delivery costs include administration costs for Missouri and overhead costs for 
Georgia. Data for South Dakota are per individual assessed. N/A indicates that the state did not provide 


information for the category 
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services provided to the participant. Data from the comparison site are relatively 
similar across the states: administration was about 6 percent to 15 percent of total costs, 
support services were 8 percent to 23 percent of total costs, and direct delivery costs 
were about 80 percent of total costs. 


In the demonstration site, however, the distribution of costs varied markedly 
among states. Administration costs were 6 percent in Texas and 20 percent in Hawaii. 
Costs of support services ranged from a low of 2 percent of total costs in Hawaii to 
over 50 percent in Georgia and Texas. 


Except in Hawaii and South Dakota, the percentage of costs devoted to direct 
delivery was lower in the demonstration site, while the percentage devoted to support 
services was higher. The higher support costs primarily resulted from higher child care 
costs in the two states with the largest relative expenditure on support services: both 
Georgia and Texas spent about 45 percent of total costs on child care in the 


demonstration site. 


As made clear by Table III-11, which displays the distribution of costs per 
participant, these differences in how the demonstration affected the distribution of costs 
are strongly related to the state’s service design and philosophy. Both Georgia and 
Texas spent large amounts on child care, $408 per participant in Georgia and $520 per 
participant in Texas, more than these states spent on direct delivery. In Texas, the 
child care costs were concentrated among participants attending postsecondary 
education or training. In contrast, child care was not an allowable demonstration 
service in Missouri, the other state with high support costs.3 Support costs in Missouri 
were mostly transportation allowances (up to $70 per week) and work-related expenses 
(up to $350 per year). Although Hawaii and South Dakota expanded the theoretical 
availability of support services for demonstration participants, the cost of support 
services per participant was actually lower in the demonstration in Hawaii and just a 
few dollars higher in South Dakota. 


3 This policy changed during the demonstration in the largest demonstration county. 
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Table III-11 
Distribution of Costs Per Participant 


South 


Georgia Hawaii Missouri Dakota Texas 


2 eee, SO Cee en oe. 


Administration N/A N/A $352 $17. N/A N/A $47 $21 $69 $38 
Direct delivery $377 $182 $1,355 $198 $772 $244 $251 $122 $515 = $377 
Support services $532 $45 $27 49 $502 $73 $13 $8 Sor 88 
Total $909 $227 $1,733 $263 $1,274 $317 $311 $141 $1,191 $454 

Childcare $408 $6 N/A N/A N/A N/A N/A N/A $520 N/A 


Note: D represents demonstration site. C represents comparison. Comparison data are from the 
comparison site, except in South Dakota, where the comparison data are from the demonstration site 
before the demonstration. (Comparison data from for Hawaii and Texas are different from those shown in 
Table III-9 because the comparison used in Table III-9 is the demonstration site before the demonstration. ) 
Direct delivery costs include administration costs for Missouri and overhead costs for Georgia. South 
Dakota data are per individual assessed. N/A indicates that the state did not provide information for the 


category. 


Hawaii spent much more than the other states on direct delivery and spent over 6 
times as much on direct delivery as did the comparison site. According to the state’s 
cost evaluation, the increased direct delivery costs were primarily the result of 
expenditures for detailed assessment and barrier removal services, although increased 
intensity for some existing E&T services also contributed to the increase in direct 
delivery costs (e.g., preemployment preparation, job search skills). Education and 
vocational training were obtained through referral to community resources and did not 


contribute much to direct delivery costs. 


Only two states, South Dakota and Texas provided information on leveraged 
funds—other community resources accessed to help serve demonstration participants. 
As shown in Table III-12, South Dakota was able to increase resources used for 
participants by 12 percent or $38 per participant by accessing community resources. 
Texas appears to have been even more successful, increasing resources by 55 percent or 
$657 per participant. These data may not be comparable across the two states. In 
South Dakota the leveraged costs came primarily from J TPA (60 percent) with the 
remainder coming from secondary education. The bulk of the leveraged funds in Texas 
(80 percent) were from “Pell grants, loans, and personal earnings.” South Dakota did 


Social Policy Research Associates III-16 





not include these funds in its count of leveraged funds. The remaining leveraged funds 
in Texas came from adult and postsecondary education and from JTPA. 


Table III-12 
Funds Leveraged by the Demonstrations 


South 
Dakota Texas 
Aggregate leveraged funds (annualized) $85,706 $507,169 
Leveraged funds per participant $38 $657 
Percent of E&T funds 12% 55% 


Note: Aggregate amounts are not comparable across states because of the wide variation in the size 
of the demonstration sites. The denominator for the Percent of E&T funds is total federal and state 


E&T costs excluding leveraged costs. Data on leveraged costs were not available for Georgia, 
Hawaii, and Missouri. 


Overall, most states increased expenditure per participant substantially to fund the 
demonstration’s enhanced service design. In most states these were very large 
increases in expenditure per participant for support costs, especially for child care. 
There were also increases in direct delivery costs per participant. But these increases 
were typically less than the increases in support costs. 
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IV. CONCLUSIONS 


INTRODUCTION 


In this chapter, we summarize the findings from the synthesis of impact and cost 
evaluations and review the findings from the Synthesis of Implementation and Process 
Evaluations (Kogan and D’ Amico, 1997) in light of the new information on 
demonstration impacts and costs. We also assess the lessons learned from the 
E&T/JOBS Conformance Demonstration in the context of the new welfare-to-work 
systems being established by states under the Personal Responsibility and Work 
Opportunity Reconciliation Act of 1996 (PRWORA). 


The E&T/JOBS Conformance Demonstration gave five states the opportunity to 
redesign the administration and operation of their Food Stamp Employment and 
Training programs by conforming E&T program regulations and services to the model 
offered by the Job Opportunities and Basic Skills (JOBS) program targeted to families 
receiving cash assistance under the federal program of Aid to Families with Dependent 
Children (AFDC). At the time the demonstrations were initiated, conformance with the 
JOBS program was intended to further the following demonstration goals: 


e Increase compliance with E&T participant requirements among 
mandatory work registrants and slow down the “revolving door” of 
curing adverse action notices by implementing more stringent 
sanctioning rules. 


e Target services to individuals most at risk of long-term dependency and 
most likely to benefit from E&T services through selective targeting 
using priority categories modeled after those used in the JOBS program. 


e Improve participant outcomes by assessing participants at the outset and 
offering individualized service assignments and service sequences 
matched to participants’ needs. 


e Improve the cost efficiency of welfare-to-work services by streamlining 
the administration of services provided to recipients of different 
assistance programs, such as AFDC, Food Stamps, and state or county 
general assistance. 


INCREASE E&T PARTICIPATION BY MANDATORY WORK REGISTRANTS 
TARGETED FOR SERVICES 


Since its inception, the E&T program has been plagued by low response rates to 
participant call-ins and low participant follow-through on assigned services. One 
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identified problem was that participants could cure sanctions at any time merely by 
demonstrating compliance at the time sanctions were announced. Thus, conformance 
with JOBS sanctioning rules was expected to improve participant compliance with 
participation requirements by establishing a mandatory waiting period after sanctioning 


before Food Stamp benefits could be reinstated. 


Also as a result of conforming to JOBS regulations, the demonstrations increased 
the size and changed the composition of the E&T mandatory work registrant pool by 
eliminating exemptions for UI recipients and women with young children between the 


ages of 3 and 6. 
As described in Chapter III, the states’ evaluations revealed that: 


e Tougher sanctioning procedures were implemented in three of the five 
demonstration sites. In Missouri, sanctioning procedures implemented 
under the demonstration were not tougher than previous E&T 
procedures. In Hawaii an expanded conciliation process resulted in 
sanctions being invoked less often. 


e Tougher sanctioning procedures implemented under the demonstration 
were accompanied by a lower rate of requested or applied sanctions in 
the three states that implemented tougher procedures. However, this 
was not necessarily the result of increased compliance among targeted 
work registrants. Texas reported reduced rates of sanctioning among 
called-in work registrants, which would be consistent with increased 
compliance. However, both Georgia and Hawaii documented declining 
participation rates among called-in work registrants under the 
demonstration. Also, as noted in the Synthesis of Implementation and 
Process Evaluations (p. IV-5), the demonstrations continued to report 
high rates of no-shows and subsequent failures to comply among those 
targeted for participation. Part of the problem appeared to be that the 
individuals referred to the E&T program as mandatory work registrants 
were often off the Food Stamp rolls or exempt by the time they were 
called in. 


e JOBS work registration rules replaced E&T work registration rules in 
all five states. As noted in SPR’s Synthesis of Implementation and 
Process Evaluations, several states made further adaptations to the work 
registration requirements implemented for the demonstrations (e.g., 
South Dakota exempted individuals with self-initiated training plans; 
Missouri exempted individuals in need of child care; and Georgia 
exempted individuals with limited transportation options). 


e The shifts in the E&T work registration rules under the demonstration 
had the effect of increasing the number of mandatory work registrants in 
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four of the five demonstration states. In two states (South Dakota and 
Hawaii) the size of the work registrant pool increased by 50% or more. 
This increase in the work registrant pool was not accompanied by an 
increase in available service funding. Thus, the increase in the size of 
the work registrant pool, in combination with the implementation of 
more individualized and more intensive E&T services—which actually 
reduced service capacity in most sites—intensified the decline in 
participation rates among mandatory work registrants that occurred in 
most demonstration states. 


e The implementation of JOBS work registration rules also changed the 
composition of the mandatory work registrant pool under the 
demonstration by causing the inclusion of more women with dependent 
children. As noted below, this shift in the characteristics of mandatory 
work registrants—in combination with increased encouragement to those 
exempt from work registration requirements to volunteer—was 
associated with substantial increases in E&T participation by young 
women with children and a sharp increase in the level of demand for 
E&T child care services. 


Although the changes implemented under the demonstration did not resolve the 
problem of high no-show and drop-out rates among E&T mandatory work registrants, 
the increased number of volunteers in a majority of the demonstration states suggests 
that E&T demonstration services were perceived as being desirable, at least by those 
Food Stamp recipients who chose to volunteer for services. The findings from the 
process evaluations also suggest at least part of the low response rate to call-ins for 
E&T services is due, not to non-compliance, but to the fact that the referral lists 
generated by Food Stamp intake workers become obsolete extremely quickly due to the 
high turnover within the Food Stamp recipient caseload. By the time Food Stamp work 
registrants are called in by the E&T program and respond (or fail to respond) to the 
call-in notices, a high percentage may no longer be subject to work participation 


requirements. 


TARGET SERVICES TO HIGH RISK AND MOTIVATED FOOD STAMP 
RECIPIENTS 


Following the model established by the JOBS program, most of the E&T 
demonstrations established priority target groups that were considered at risk of long- 
term Food Stamp dependency. As encouraged under the JOBS program, most of the 
E&T/JOBS conformance demonstration projects also gave priority to volunteers, 
including both “mandatory volunteers” (mandatory work registrants who volunteer for 
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participation in E&T services before being called in) and “exempt volunteers” 


(individuals not subject to mandatory work registration). 


As described in Chapter 3, the states’ evaluations revealed that: 


e Selective targeting procedures were implemented as planned in all 
states. Some states called in all work registrants belonging to target 
groups before calling in any non-target group members. Other states 
called in a mixture of target group and non-target group work 
registrants at the same time. 


e Selective targeting generally led to greater representation of priority 
groups among E&T participants. 
e However, the absolute number of E&T participants belonging to priority 


groups fell in several states because the decline in overall participation 
rates overwhelmed the effect of selective targeting. 


e Encouragement of volunteers led to substantial participation by 
volunteers in most states. 


e Selective targeting and encouraging volunteers had countervailing 
effects on participant characteristics. Targeting to priority groups was 
supposed to increase services to individuals who were most at risk of 
long-term Food Stamp dependency and thus, needed services the most. 
In contrast, the JOBS policy of priority to volunteers was based on a 
desire to help individuals who were already motivated to improve their 
skills. In the case of the E&T demonstration, volunteers tended to be 
more educated individuals who were interested in advancing their 
education and training. Thus, in the majority of states, the success of 
the demonstration projects in recruiting volunteers had an unintended 
effect on overall participant characteristics: it decreased the percentage 
of total participants who were high school dropouts and increased the 
percentage who were white. 


The use of selective targeting in combination with individualized service 
assignments and sequences was intended to focus limited E&T resources on helping 
work registrants with substantial employment preparation and skills enhancement needs 
enter community education and training programs that could help address their needs. 
(In general, studies have found that the net impacts of participating in employment and 
training services are greatest for individuals with the most limited skills upon program 
entry.) However, because they did not limit support for classroom training and 
associated supportive services to individuals with the lowest levels of existing skills, the 
demonstrations ended up serving a number of individuals with a relatively high level of 
skills (e.g., high school graduates) who were interested in advancing their education. 
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An undetermined portion of these participants might have been able to attend training 
without program support. As a result, several of the demonstration projects (Texas and 
Georgia) devoted a relatively high proportion of project resources to providing child 
care and other supportive services to individuals already motivated to attend—or, in 
some cases, already attending—education and vocational training at the time they 
enrolled in E&T services. 


MATCH SERVICES TO PARTICIPANTS’ NEEDS AND IMPROVE 
PARTICIPANT OUTCOMES 


Program design changes that were expected to improve the effectiveness of E&T 
services for a wide range of clients included: (1) replacement of a “one-size-fits-all” 
sequence of services with a more diverse menu of services and individualized service 
planning; (2) redesign of the content of job readiness/job search assistance to help 
improve participants’ self-knowledge, self-esteem, and practical job search skills; (3) 
encouragement of participants with limited educational skills to attend basic skills 
training and complete their high school diplomas; and (4) availability of an enhanced 
menu of supportive services to support classroom training, including a higher rate for 
reimbursement of child care and transportation expenses. 


The state impact evaluations confirm that the demonstrations increased the range 
and intensity of education and training services provided to E&T participants. 
Specifically, the impact evaluation results confirm that the demonstration projects 
increased the frequency with which E&T participants received assessment, education 
and vocational training services, and participated in work experience or community 
service activities. The frequency of utilization of individual job search and job search 
skills training declined, as these services began to be matched to individual participants’ 
needs, rather than assigned to all participants as a required first service. 


Despite the overall success in implementing more intensive services, the state 
impact evaluations provide evidence of at most modest improvements in participant 
outcomes. Three of the five states showed increased employment among demonstration 
participants. Because these analyses did not adjust for demographic or economic 
differences between the demonstration and the comparison, they are not conclusive. 
Indeed, the remaining two states found no significant employment effect after 
controlling for participant characteristics and economic conditions, even though there 
were positive effects in simple difference in means analyses. Further, the effects of the 
sharply reduced participation rates in many states may well have offset these modest 
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effects. That is, the demonstrations may have had negative effects on the many work 
registrants who did not receive services under the demonstration but would have 


received services under the nondemonstration E&T service design. 


Moreover, the two state evaluations that examined Food Stamp receipt found 
mixed effects that, if anything, indicate that the demonstration may have increased Food 
Stamp utilization during the first year after participants entered the E&T program. 


IMPROVE THE CosT-EFFICIENCY OF WELFARE-TO-WORK SERVICES 

One possible result of conforming E&T operations to the JOBS model would have 
been for states to consolidate administration and operations of the E&T and JOBS 
programs at both the state and county levels to reduce duplication of effort and save on 
administrative costs. Another goal was to increase the ability of the E&T program to 
leverage community education and training resources on behalf of E&T participants as 
a result of improved access to interagency linkages initiated by JOBS services. 


In fact, only two of the five demonstration states (Texas and Georgia) 
consolidated E&T and JOBS operations at the county level, and only Texas instituted 
integrated caseloads for local case management of JOBS and E&T clients. 
Demonstration states were not generally able to identify how administrative costs were 
affected by the demonstration, but cost evaluation findings clearly show that overall 
E&T program costs increased dramatically (between 43% and 368 %) as a result of the 
demonstration and that per participant costs sky-rocketed, due to the substantial 
reduction in participant volumes associated with the demonstrations. 


LESSONS LEARNED FROM THE CONFORMANCE DEMONSTRATIONS 

As noted in SPR’s Synthesis of Implementation and Process Evaluations, the 
experiences of the demonstration states suggest that regulatory conformance among and 
administrative consolidation of different welfare-to-work programs is administratively 
feasible at both the state and local levels. This bodes well for the potential success of 
states choosing the Simplified Food Stamp Program option of consolidating 
administration of Food Stamps and Temporary Assistance to Needy Families (TANF) 
under the Personal Responsibility and Work Opportunity Reconciliation Act of 1996 
(PRWORA). 
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However the state impact evaluations raise serious questions about whether 
conformance—as implemented in the E&T/JOBS Conformance Demonstration—leads to 
improved outcomes for E&T participants. 


One of the lessons suggested by the E&T/JOBS Conformance Demonstration is 
that client targeting criteria and service approaches established for work programs for 
AFDC recipient households should not be uncritically transferred the Food Stamp E&T 
program with its diverse caseloads and rapid caseload turnover. 


Targeting services to women with young children and providing support services 
to volunteers were part of a carefully crafted strategy under JOBS to target services to 
motivated individuals and individuals at risk of long-term welfare dependency. In 
several demonstration states, policymakers adopted these JOBS targeting strategies for 
the E&T work registrants under the conformance demonstration without tailoring them 
to the different characteristics of E&T work registrant caseloads. These targeting 
policies—which caused several states to shift E&T attention and resources away from 
single individuals and individuals with limited basic skills toward women with children 
and individuals interested in furthering their post-high-school education—were not 
necessarily well thought-out in terms of their effects on the overall E&T work 


registrant population. 


Another JOBS targeting principle that had been crafted for application to AFDC 
households called for selective participation emphasizing those most at risk of long-term 
welfare recipiency. However, the result of applying this principle to the E&T work 
registrant pool was that a large number of job-ready Food Stamp recipients in 
demonstration states with highly selective participation designs (especially Missouri) 
were no longer required to participate in job-search activities to qualify for receipt of 
Food Stamp benefits. Because the demonstration evaluations typically estimated 
outcomes only for participants, they did not give information about how selective 
targeting of services to a limited number of work registrants affected outcomes for 
those work registrants no longer required to participate. However, it is likely that to 
improve overall outcomes across the entire work registrant pool, E&T programs will 
have to distribute resources across a larger number of participants than did some of the 


conformance demonstration projects. 


The conformance demonstration evaluation results showed that selective targeting 


to priority groups could increase services to those groups. However, they also suggest 


IV-7 Social Policy Research Associates 


that providing supportive services, including child care assistance, to individuals in 
education and training activities may induce them to stay on Food Stamps longer, as 
they continue those activities. At this point, one can only speculate about whether 
increased Food Stamp receipt in the short run will be followed by reduced dependency 
in the long run as these individuals complete their training and obtain employment. 


The Food Stamp E&T program faces a number of new challenges in the coming 
months as a result of changes to the Food Stamp and cash assistance systems under 
PRWORA and the Balanced Budget Act of 1997. First, the E&T program faces the 
challenge of serving able bodied adults without dependent children (ABAWDs) who are 
at risk of losing Food Stamp eligibility after three months unless they work at least 20 
hours a week. Additional funds have been already been appropriated by Congress to 
ensure that the E&T program will be able to assist these at-risk Food Stamp recipients 
to find employment, or if jobs are not available, to perform a workfare activity that will 
allow them to retain eligibility for Food Stamps. 


While consistent with the “work first” approach being emphasized for TANF 
recipients, Food Stamp E&T services to ABAWDs require the development of distinct 
service designs and service delivery arrangements—e.g. placing participants into 
workfare assignments—because of the extremely short eligibility time limits for these 
individuals unless they obtain employment. The requirements in the Balanced Budget 
Act of 1997 require that 80% of federal E&T funds must be used to serve ABAWDs 
who are placed in and comply with a qualifying work, training, or workfare program. 
Thus, at least in the short run, the emphasis of E&T program designs will need to be on 
providing cost-effective mechanisms for ABAWDs to remain eligible for Food Stamp 
benefits when jobs are not available in the local labor market. 


Over time, however, the E&T program may inherit increased responsibility for 
poor households with dependent children who lose cash assistance eligibility as a result 
of exhausting state or federal time limits on household receipt of cash assistance. As 
long as the requirement to spend the vast majority of E&T funds on ABAWDs remains 
in place, the E&T program’s ability to serve this population may be limited. To the 
extent that resources become available in the future to address the varied employability- 
development needs of these high-need households, states will need to develop service 
designs that are flexible enough to respond to widely varying client needs. The 
examples of selective targeting, individualized service planning, and varied service 
offerings tested under the E&T/JOBS Conformance Demonstration will provide a 
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starting point for states interested in designing E&T services and operations to meet the 
long-term employability-development challenges posed by welfare reform. 
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APPENDIX A 
THE EVALUATION OF THE GEORGIA JET 
DEMONSTRATION 


THE EVALUATION OF THE GEORGIA JET 
DEMONSTRATION 


The Georgia conformance demonstration (called JET, for the “JOBS Employment 
and Training Program”) was established under the umbrella of the state’s PEACH 
(“Positive Employment and Community Help”) program, which also included 
employment and training services under E&T and JOBS. The demonstration was 
established to further three goals: 


e Facilitate the coordination of comprehensive training and skills 
development for public assistance recipients. 


e Provide necessary support services to public assistance recipients 
participating in training and making the transition to employment. 


e Assure that public assistance clients have real opportunities for job 
placement and career development. 


To further these goals, Georgia undertook several changes simultaneously. As 
described below, it implemented a statewide redesign of the E&T program to bring it 
closer to the JOBS model. Within the four demonstration counties, demonstration 
approval made it possible for Georgia to enhance the range and level of supportive 
services available to demonstration participants. Doing so, it was believed, would 
enable participants to undertake the education and training they need to obtain 
employment and reduce dependency on Food Stamps. The demonstration also resulted 
in the provision of individual service planning and case management that was more 


intensive than was typical for E&T clients in non-demonstration counties. 


SPR’s process and implementation report provides additional information 
concerning the implementation of the demonstration project (including its goals and 
objectives, its history and evolution, program costs, and other features). This appendix 
summarizes the state’s impact and cost evaluations, including an overview of the 
evaluation design and a discussion of the estimated impacts on participation patterns, 


services, outcomes, and costs. 


EVALUATION DESIGN 

To evaluate the impacts of the demonstration, Georgia implemented a comparison 
site design. For each of the counties included in the demonstration, a matched 
comparison county was selected. As shown in Table A-1, paired counties were 
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generally similar in terms of the size of the Food Stamp Program, the ethnicity, age, 
and gender of Food Stamp Program participants; the number of work registrants; and 


the local unemployment rate. 


Table A-1 
Comparison of Demonstration and Comparison Counties 


Chatham Richmond | Clayton Cobb Gilmer Fannin Glynn _ Spalding 


Demo. Comp. Demo. Comp. Demo. Comp. Demo. Comp. 
FSP households 10,007 10,836 5,369 5,169 666 658 oD ea 25) 
Percent Black 84% 77% 47% 42% 0% 0% 65% 67% 
Age 18-34 28% 28% 29% 28% 28% 29% 28% 28% 
Percent female 60% 60% 60% 59% 55% 54% 59% 58% 
Work registrants 2,397 Sez T2 3,596 2,480 394 396 521 797 
Unemployment rate 4.3% 4.9% 5.8% 4.1% 6.2% 6.7% 4.2% 5.8% 


Source: Appendix to Georgia’s Implementation and Process Findings. 


Although the demonstration and comparison counties are not matched exactly, the 
difference between matched pairs is small relative to the variation over all counties. 


In addition, Georgia implemented a random assignment design within the 
demonstration to test the impact of expanded supportive services. The treatment group 
received enhanced supportive services according to the JOBS supportive services plan, 
while the control group received the same supportive services to which E&T 
participants in the comparison counties were entitled. 


The comparison site design relies on participants in the demonstration and 
comparison site being similar. Georgia used two strategies to help assure that their 
impact estimates were not confounded by differences between the demonstration and 
comparison sites. First, the demonstration sites were matched with similar comparison 
counties, as discussed above. Second, multivariate regression-type models were 
estimated to control for measured differences between the demonstration and 
comparison samples. Because of data limitations in the comparison counties, however, 
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only a relatively few characteristics could be used in these models, including county 
pair, age, and volunteer status.! Because of the limited number of control variables and 
because the models were not estimated for all outcomes, the use of matched county 
pairs was the primary strategy used to control for differences between the 
demonstration and comparison samples. 


Several other limitations should be kept in mind when assessing the estimated 
impacts of the demonstration. First, the demonstration counties tended to call in those 
individuals who were most recently classified as work registrants, while comparison 
counties called in those who had been classified work registrants the longest. As a 
result, work registrants that could quickly find a job on their own tended to be included 
in the demonstration samples, but excluded from the comparison samples. In addition, 
in the comparison counties, individuals were defined as participants if they reported to 
the first face-to-face meeting with a case manager after being called in for participation. 
In contrast, in the demonstration counties, individuals were not considered a participant 
until they attend both an initial orientation meeting and a face-to-face meeting with a 
case manager. Thus individuals became participants at their first activity, in 
comparison counties, but at their second activity in demonstration counties. As a result 
individuals in the demonstration samples may be more motivated. The demonstration 
samples, therefore, may differ from the comparison samples in ways that lead to higher 


outcomes, even if the demonstration is ineffective. 


Second, outcomes are measured both at exit from the E&T program and about 1 
year later. However, the demonstration tended to continue serving people until they 
achieved a positive outcome, while individuals could exit from the comparison 
counties’ E&T programs after completing required activities, but before achieving a 
positive outcome. As a result the analysis of outcomes at exit may be more favorable to 
the demonstration than would an analysis of outcomes at some fixed time after the start 
of E&T services. The average length of participation, however, was only a little higher 
in the demonstration: 110 days as compared to 96 days in the nondemonstration 
counties. Therefore, the bias is unlikely to be large. Further, outcomes measured a 
year after exit should be virtually unaffected by this potential bias. 


1 Some models also included receipt of services or number of days enrolled. 
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At the outset of the demonstration period, statewide changes in the E&T program 
reduced the differences between the demonstration and comparison service designs. 
For instance, the E&T program also moved from a nearly total reliance on job search to 
the availability of a wide variety of activities including initial assessment, education, 
vocational skills training and work experience. It is important to note, however, that 
only a small percentage of nondemonstration participants took part in these new E&T 
activities, whereas a fairly large number of individuals participated in the activities 
offered in the demonstration. Thus, the demonstration sites provided substantially 
more intensive services, despite the changes in statewide E&T program. The estimated 
impacts of the demonstration should be understood as the results of the remaining 
differences between the demonstration and comparison sites, which included 1) the 
ability of exempt individuals to volunteer for participation in the demonstration sites, 2) 
the availability of a wider range of supportive services, 3) the implementation of JOBS 
work registration rules which required participation by individuals with children 
between 3 and 6 years of age and 4) a greater emphasis on individualized assessment 


and service sequences in the demonstration sites. 


IMPACT FINDINGS 
Participation Patterns 
Work Registrants 
By reducing exemptions from work registration requirements, the demonstration 
was expected to increase the number of work registrants. Georgia’s impact report did 
not report impacts on the number of work registrants. However, its process and 
implementation report reported some information on work registration, as displayed in 


Table A-2. 


Table A-2 
Work Registration in Demonstration and Comparison Counties 
Demonstration Comparison 

Before demonstration (1992) 7,908 6,945 
During demonstration 

4/93 to 3/94 8,048 4,887 

4/94 to 3/95 6,871 TEES! 
Average 7,495 6,012 


Source: Appendix to Georgia’s Implementation and Process Findings. 
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As shown in the table, the total number of work registrants in the demonstration 
counties was nearly unchanged in the first year of the demonstration, but declined in the 
second year. In contrast the number of work registrants declined at first in the 
comparison counties and then increased. Overall, work registration decreased less in 
the demonstration counties than in the comparison counties. Thus, these data are 
consistent with a small demonstration-induced increase in work registration of about 9 


percent, although the evidence is not definitive. 


Participation Rates 
Because the demonstration both targeted services to priority groups and was 
intended to provide more intensive services to those who participated, it was expected 
that participation rates would decline. By participation we mean an individual who 


received at least some minimal service from the E&T program.’ 


Georgia’s impact evaluation examined participation rates among mandatory work 
registrants and volunteers who were called in for service. The descriptive statistics 
shown in Table A-3 indicate that the participation rate among those called in was 15 
percent in the demonstration counties as compared to 20 percent in the comparison 
counties. The participation rate for mandatory work registrants was 13 percent in the 
demonstration; it was 29 percent for volunteers. The overall participation rate (among 
all work registrants), however, also depends on the call-in rate, which was set to match 
caseloads to service capacity at each site. However, a comparison of the data in tables 
A-2 and A-3 shows that there were many fewer participants in the demonstration 
counties than in the comparison counties despite the fact that the number of work 
registrants tended to be somewhat higher in the demonstration counties.* Thus, the 
overall participation rate must have been lower in the demonstration counties. 


2 Georgia’s impact report refers to these individuals as “enrollees,” who are defined as individuals 
who reported to a reportable program component. The demonstration counties did not treat the group 
orientation as a reportable component. 


3 The data in Tables A-2 and A-3 are not comparable because they cover different time periods. 
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Volunteers 
The demonstration appears to have succeeded in targeting services to volunteers: 
28 percent of participants in the demonstration site were volunteers while the 
comparison sites did not serve volunteers.4 Further, the percentage of called-in 


Table A-3 
Participation in Demonstration and Comparison Counties 
(July 1993 to June 1995) 


Demonstration Comparison 


Called-in work registrants and 8,027 10,989 
volunteers 

Participants 1,182 22212 
Participation rate among those 15% 20% 
selected 


Source: Georgia’s Impact Report, Table 7, p. 12, and Table 15, p. 25. 


volunteers who participated was more than twice as high (29 percent) as the percentage 
of called-in mandatory registrants who participated (13 percent). Moreover, compared 
to mandatory work registrants, volunteers had a substantially different demographic 
profile—they were nearly all female, more highly educated (76 percent were high 
school graduates versus only 42 percent for mandatory participants), had virtually no 
homeless exemptions (1 percent versus 25 percent for mandatory participants), and on 
average were 8 years younger than mandatory work registrants who participated. 


Priority Groups 
Georgia identified six priority groups: 


e Custodial parents under age 24 who have not completed high school or 
equivalent and are not enrolled in school, or who have less than 6 
months of work experience. 


e Primary wage earners in two-parent households. 
BE. 2 A 


4 In Georgia volunteers could be either exempt from work registration or mandatory work 
registrants. No distinction was made between these two groups of volunteers. 
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e Persons who have received Food Stamps for 36 of the previous 60 
months. 


e Self-initiated participants (those already enrolled in education or 
vocational training). 


e Registrants without a high school diploma or GED. 


e Registrants with less than six months work history. 


Because of data limitations, Georgia could not compare the levels of service to 
priority groups between the demonstration and comparison counties. However, all but 
the lowest priority group had higher participation rates among those called in for 
services than individuals who were not members of any priority group, as shown in 
Table A-4. 


Table A-4 
Participation Rate by Priority Group 
(In Order of Priority) 
Demonstration 
Teenage parents 18% 
Primary wage earners 23% 
Long-term Food Stamps recipients 21% 
Recipients in self-initiated training 28% 
Limited education 12% 
Limited work history 9% 
Not a priority group member 11% 


Source: Georgia’s Impact Report, p. 22. 


In addition, the impact report provides an analysis that compares participants with 
nonparticipants within the demonstration. A logistic regression model shows that 
membership in four of the six priority groups was associated with higher probabilities 
of participation (among called-in registrants): teen parents, primary wage earners, long- 
term recipients, and self-initiated participants (individuals who were enrolled in training 
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at the time of referral).5 Other factors that increased the probability of participation 
were married with spouse present, being older, black, female and volunteer status. The 
two priority groups with the lowest priority were not positively related to participation: 
limited education was unrelated to the probability of participation while limited work 
history, the group with the lowest priority, was associated with reduced probability of 
participation (relative to individuals not in any priority group). Overall, however, the 
demonstration did appear to focus services on those groups with the highest priority. 


To some extent, the targeting to priority groups and the emphasis on serving 
volunteers had offsetting effects on the overall characteristics of participants. 
Compared to nonparticipating called-in work registrants, participants were more highly 
educated (71 percent high school graduates versus 46 percent) and had a higher 
percentage of female participants (74 percent versus 56 percent), probably because 


volunteers were mostly female and high school graduates. 


Because these analyses were based on comparing participants and nonparticipants 
within the sample of work registrants targeted for participation by each site, the 
demonstration was probably even more successful in targeting priority groups than 
implied by these results to the extent that the call-in process itself targeted members of 
priority groups. In the absence of data from the comparison site, we do not know if the 
demonstration was more successful than the comparison counties in serving these 


groups. 


In summary, the Georgia demonstration tended to serve fewer participants but 


appears to have been generally successful in targeting services towards volunteers and 
priority groups. 


Sanctioning 
The demonstration implemented stricter sanctioning provisions so that sanctions 
were more difficult to cure. These stricter penalties could have led either to increased 
sanctioning or greater compliance. Among selected work registrants, a significantly 
lower percentage had been sanctioned at exit from the program in the demonstration (21 
percent) than in the comparison counties (36 percent). The state’s evaluation suggests 
that this reduction was due to an increase in individual exemptions from participation 


requirements—the reduction in sanctions measured at exit from the program was 


i 


5 Georgia’s Impact Report, Table 13, p. 22. 
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accompanied by increased exemptions: 43 percent of called-in registrants were 


exempted in the demonstration as compared to 29 percent in the comparison counties. 


Among participants (those who participated in at least some activities), the 
sanctioning rate was similar in the demonstration and comparison counties—about 22 
percent. 


SERVICES RECEIVED 

Types of Services 

Georgia’s JET demonstration emphasized participant access to a more flexible 
sequence of component services, in combination with individualized assessment and 
one-on-one case management for all clients and more generous supportive services (for 
the randomly assigned treatment group). Table A-5 shows the incidence of services in 
the demonstration and comparison counties. 


Table A-5 
Service Receipt by Participants 
(July 1993 to June 1995) 


Demonstration Comparison 
Assessment 97% 17% 
Job Search—Group 1% 77% 
Job Search—Individual 39% 75% 
Secondary Education 3% 0% 
Adult Education 23% 2% 
English as a Second Language 1% 1% 
Postsecondary Education 10% 2% 
Job Readiness Training 2% 6% 
Occupational Skills Class 3% 0% 
Work Experience 5% 0% 
Employment 7% 5% 
Number of Enrollees 1,182 D3 


Source Georgia’s Impact Report, Table 15, p. 25 
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Participants in the demonstration were much more likely to receive assessment 
and much less likely to receive group job search or individual job search than 
participants in the comparison counties. Further, they were considerably more likely to 
participate in educational activities, such as secondary education, adult education and 
postsecondary education. A small percentage received occupational skills training and 
work experience, which were not available in the comparison site. The evaluation also 
found that younger participants were more likely than were older participants to receive 
educational activities. Educational activities were typically funded by other programs, 
either Pell grants for postsecondary education or Adult Education and English as a 
Second Language courses available in the local area. Although Table A-5 is based on a 
simple difference in means analysis, it is unlikely that differences in participant 
characteristics between the demonstration and comparison counties could explain the 
dramatic differences in the types of services received by participants. Some of the 
enrollment in educational services, however, is explained by the presence of volunteers, 
many of whom had started the educational activity before participation in the 
demonstration. For example two-thirds of participants in postsecondary education were 


volunteers. 


In both the demonstration and comparison counties, a substantial percentage of 
participants did not receive services beyond their initial activity: 28 percent of 
participants in the demonstration exited the E&T program after assessment, while 21 
percent exited after the first activity (typically group job search) in the comparison 


counties. 


In summary, the demonstration was successful in increasing individualized 
assessment and in providing (directly or through referral) a wider range of services 


with a focus on educational activities, relative to comparison counties. 


Intensity of Service 


Georgia did not provide information on the intensity of services provided. 


OUTCOMES 

Employment and Earnings 

The Georgia impact evaluation examined employment rates and earnings 9 to 12 
months after termination among called-in work registrants (not just participants). The 


results are shown in Table A-6. 
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These data show that both average earnings and the employment rate were higher 
in the demonstration than in the comparison counties. Average earnings of those 
employed, however, were somewhat lower in the demonstration counties. Thus, the 
increase in overall average earnings is due entirely to the increase in employment. 


Table A-6 
Employment and Earnings One Year After Program Exit 
for Called-In Work Registrants 
(July 1993 to June 1995) 


Demonstration Comparison 
Average earnings (all selected $902 $839 
registrants) 
Percent employed 43% 36% 
Average earnings (employed) $2,106 $2,314 


Source: Georgia’s Impact Report, Table 26, p. 38. 


Further analysis showed that the difference in overall average earnings was 
concentrated among participants—there was little difference in average earnings for 
work registrants who were called in but did not participate. Thus, the increase in 
earnings and employment likely resulted from the services received by participants. 


The Georgia Impact Report also demonstrated that the expanded supportive 
services provided to a random subgroup in the demonstration sites increased both 


employment and earnings. 


Food Stamp Receipt 

A primary goal of the demonstration was to increase the number of individuals 
who leave the Food Stamp Program as a result of increased employment and earnings. 
The state’s evaluation found that a greater percentage of participants were off Food 
Stamps at exit from E&T in the demonstration (20 percent) than in the comparison 
counties (10 percent). Demonstration participants were also more likely to be still on 
Food Stamps, but employed, upon exit from E&T (17 percent versus 12 percent).® 


6 Multivariate logistic models confirmed that there were significant differences even after 
controlling for age, the number of days enrolled, and volunteer status. 
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These effects may to some extent be an artifact of the demonstration design: the 
demonstration tended to continue to serve individuals until they became employed or 
left Food Stamps, while in the comparison counties many participants completed their 
activities and exited from E&T without achieving successful employment outcome. 


A comparison of outcomes between the treatment and control groups within the 
demonstration shows, however, that the provision of enhanced supportive services 
reduced the percentage off Food Stamps at exit and increased the percentage on Food 


Stamps, but employed, at exit. 


A year after exit from E&T, however, the situation among selected work 
registrants (not just participants) was reversed: 61 percent of selected registrants in the 
demonstration were off Food Stamps while 72 percent were off Food Stamps in the 


comparison counties.’ 


This apparent reversal of the impact of the demonstration on Food Stamps receipt 
is puzzling, but might be explained by the lower participation rate and higher 
exemption rate in the demonstration sites compared to the comparison sites. Overall, 
the evidence on the impacts of the demonstration on Food Stamps receipt is 
contradictory. But it appears that the demonstration actually may have increased Food 


Stamps receipt among work registrants one year after exit from the E&T program. 


Costs 

Although the demonstration and comparison sites were similar in size, Georgia 
spent about twice as much in the demonstration sites, $537,295 per year as compared to 
$251,193 in the comparison sites. Costs per participant, however, were four times as 
high in the demonstration sites ($909 compared to $227) because the doubled 
expenditures were used to serve fewer participants in the demonstration sites. 


Table A-7 displays the distribution of costs in the demonstration and comparison 
counties. Both the demonstration and comparison sites spent the same amount on 
program staff, who provided assessment, job search, and job readiness services. These 


costs comprised 80 percent of all costs in the comparison site. The demonstration, 





7 These data are based on a sample of selected work registrants that was not designed to replicate 
the full samples used for the evaluation—it underrepresented participants and overrepresented 
nonparticipants. Evidence was presented, however, to show that outcomes at exit for the sample were 
similar to those for all selected work registrants. 
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however, spent considerable additional funds on supportive services—58 percent of all 
costs or $532 per participant. The bulk of these costs were for child care, which 
amounted to $408 per participant in the demonstration, but only $6 per participant in 
the comparison site. Thus, the major difference in costs between the sites was the 
considerable expenditure on child care. In addition, the case managers in the 
demonstration site provided more intensive services to fewer participants, so the cost of 
direct service delivery was $341 per participant in the demonstration, but only $182 per 
participant in the comparison sites. 

Table A-7 


Distribution of Costs 
(July 1993 to June 1995) 


Demonstration | Comparison 
Total Cost (Annualized) 
Direct service delivery including case $201,300 $201,300 
management 
Tuition reimbursement $21,850 $0 
Supportive services $314,146 $49,893 
Child care assistance $241,328 $7,016 
Total $537,295 $251,193 
Percentage Distribution 
Direct service delivery including case 37% 80% 
management 
Tuition reimbursement 4% 0% 
Supportive services 58 % 20% 
Child care assistance 45% 3% 
Cost Per Participant 
Direct service delivery including case $341 $182 
management 
Tuition reimbursement $37 $0 
Supportive services $532 $45 
Child care assistance $408 $6 
Total $909 $227 


Source: Georgia Impact Report, Table 15, p. 25, Table 35, p. 56, and Table 37, p. 58. 
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SUMMARY 


The major findings from the Georgia evaluation are that the demonstration: 


Increased individualized assessment. 
Provided a wider range of services to fewer participants. 


Increased participation in educational activities substantially in the sharp 
reductions in job search and job search skill training. 


Led to increased employment, both at exit from E&T and one year 
later, but served a more motivated clientele. 


May have led to increased Food Stamps receipt one year after exit 
despite leading to a reduction in Food Stamps receipt at exit. 


Spent much more per participant on child care and about four times as 
much overall per participant than did the comparison site. 
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APPENDIX B 
THE EVALUATION OF THE HAWAII CONFORMANCE 
DEMONSTRATION 


THE EVALUATION OF THE HAWAII CONFORMANCE 
DEMONSTRATION 


OVERVIEW 


Hawaii’s Food Stamp Employment and Training/JOBS Conformance 
Demonstration was in operation on the island of Oahu between November 1993 and 
September 1996. Demonstration objectives included: 


e Reducing administrative and service costs due to coordination across 
programs. 


e Enhancing E&T services through the use of a new case management 
system, the addition of new barrier removal and family social support 
services, and an expanded emphasis on educational activities. 


e Decreasing program errors due to reduced complexity of and conflicts 
between program regulations for the E&T and JOBS programs. 


The demonstration was intended to increase overall fairness by offering all public 
assistance recipients “the same realistic and meaningful opportunities to achieve self- 
sufficiency.” The key features of the demonstration’s service strategy included the 
introduction of comprehensive case management designed to link families and 
individuals to needed support services for the removal of psycho-social barriers to 
employment, followed by the provision of needed employment preparation training, 


basic education, and vocational training services. 


SPR’s process and implementation report provides additional information 
concerning the implementation of the demonstration project (including its goals and 
objectives, its history and evolution, program costs, and other features). This appendix 
summarizes the state’s impact and cost evaluations, including an overview of the 
evaluation design and a discussion of the estimated impacts on participation patterns, 


services, outcomes, and costs. 


EVALUATION DESIGN 

Hawaii chose to use a combined comparison site/pre-post design to evaluate the 
impacts of its demonstration. The demonstration was operated in Oahu, the most 
populous of the Hawaiian Islands, which contains about 70 percent of the state’s Food 
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Stamp recipients. The state selected the Island of Hawaii (the “Big Island”) as the 
comparison site.! There are substantial differences in the populations and economies of 
Oahu and the Big Island. The Big Island’s economy is much more dependent on 
agriculture than is Oahu; it also has nearly twice the poverty rate and higher 


unemployment rates. 


Because of these substantial differences, the state’s evaluators were not willing to 
rely on simple comparisons between the demonstration and comparison sites to 
determine the impacts of the demonstration. Instead, the estimate of the demonstration 
impact was based on the difference between the demonstration and comparison sites of 
the change that occurred from before to during the demonstration period. For example, 
if earnings increases by $15 in the demonstration site and increases by $5 in the 
comparison site, then the estimate of the impact is $10 (15 - 5). 


The strength of the combined design is that it adjusts both for preexisting 
differences between the demonstration and comparison sites and for trends over time 
that are common to both sites. Thus, the main threat to the validity of the estimates is 
differential changes in participant characteristics or economic conditions. To address 
this possibility, the state’s impact evaluation estimated regression models that controlled 
for a variety of participant demographic characteristics, including household size, 
gender, age, U.S. citizenship, ethnicity (dummy variables for Filipino, Hawaiian, 
white, and mixed ancestry), high school graduate, highest school grade completed, and 
marital status. Models for service and other in-program dependent variables also 
included two economic variables, employment growth, and the unemployment rate. 
Because there is only one demonstration and one comparison county, these county-level 
economic variables are likely to be highly correlated with the variable measuring the 
demonstration impact. This correlation may have seriously reduced the statistical 
power of the regression models. Models for labor market outcomes, however, did not 
include these economic conditions. Thus, those estimates could be affected by 
differential economic trends between the demonstration and comparison sites. 


Hawaii implemented “work first” requirements in the JOBS program beginning in 
April 1995. These requirements, which were introduced into the demonstration as 
well, required all participants, including those in education and training components, to 





1 The term ‘Hawaii’ is used in this Appendix to refer to the state of Hawaii, not the comparison 
site of the same name. 
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work at least 8 hours per week and preferably 16 hours per week. In addition, a 
dramatic reduction in state funding led to a redesign of the demonstration service 
delivery design in July 1995—program emphasis shifted from barrier removal and 
employability development to immediate employment. To avoid the effects of these 
changes, the impact evaluation covered the period from January 1994 to June 1995. 
Although most of the evaluation period occurs before the program changes, there is 
some evidence that the work first requirements may have induced demonstration 
participants to drop out of training and education activities near the end of the 
evaluation period. The demonstration findings, therefore, should be interpreted with 
this qualification in mind. 


IMPACT FINDINGS 

Participation Patterns 

Work Registrants 

The demonstration implemented the JOBS program’s more inclusive work 
registration rules and was, therefore, expected to increase the number of work 
registrants. Hawaii’s impact report did not provide information on the total number of 
work registrants.2 The process and implementation report, however, indicated that 
there was a substantial increase (by over 100 percent) in the number of work registrants 
in the demonstration site. Because total Food Stamp participation increased by a 
smaller amount (by only 38 percent from FY 92 to FY 96), it seems likely that the 
demonstration did result in increased work registration among Food Stamp recipients-— 


probably by more than 60%. 


Participation Rates 
Because the demonstration was intended to provide more intensive services to 
priority groups, it was expected that the participation rate would decline. The Hawaii 
evaluators did not have data on the number of mandatory work registrants. Instead, 
they reported on the number of participants out of those who were called-in. Total call- 
ins were about the same in Oahu before and after the demonstration, about 1,600. As 


2 Center for the Study of Human Resources, University of Texas at Austin, Hawaii Food Stamp 
Employment and Training/JOBS Conformance Demonstration: Impact Evaluation Final Report, June 
1997. 


3 State of Hawaii, Department of Human Services, E&T/JOBS Conformance Demonstration 
Project: Implementation and Process Report, May 1996, p. 45. 
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shown in Table D-1, most registrants who were called in before the demonstration (88 
percent) went on to participate in a training component. But during the demonstration, 
only 527 (32 percent) actually participated, a difference of 56 percentage points. In 
comparison, there was only a small decline in participation in the comparison site (from 
94 percent to 86 percent). Regression models that control for demographic and 
economic differences confirm that the demonstration reduced the participation rate by 


58 percentage points.* 


Table B-1 
Participation Rate in the Demonstration and Comparison Sites 
(Among Work Registrants Called In) 


Demonstration Effect 
Demonstration Comparison (percentage points) 


During demonstration 32% 86% 
Before demonstration 88% 94% 
Change -56% -8% -48 % 


Source: Hawaii Food Stamp Employment and Training/JOBS Conformance Demonstration: Impact 
Evaluation Final Report, Table 4, page 12. 


Because the number of call-ins did not change from before the demonstration, the 
decline in the participation rate more than offset the increase in work registrants. Thus 
the number of work registrants declined from an average of 973 per year before the 


demonstration to 351 during the demonstration. 


Volunteers 
Data on participation by volunteers were presented in the states’ implementation 
and process report (p. 31). Despite the larger size of the demonstration site, it served 
many fewer volunteers: only 4 percent of participants in the demonstration were exempt 
volunteers as compared to 32 percent in the comparison site. Another 58 percent were, 





4 Some of the difference in participation rates may result in differences in data collection 
procedures. In the comparison site a person is counted as a participant if enrolled in a program 
component; in the demonstration a person is counted as a participant only if at least one hour is recorded 


as completed. 
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however, Classified as mandatory volunteers because they volunteered before being 
called in. Thus, the Hawaii demonstration served few exempt volunteers, but many 
mandatory volunteers. 


Priority Groups 
As mentioned above, as part of the demonstration, Hawaii intended to focus 
services on several priority groups. Hawaii’s impact report did not examine the 
characteristics of participants. The state’s process and implementation report, however, 
reported the incidence of priority groups among demonstration participants, as shown 
in Table B-2. No comparison data are available. 


Table B-2 
Priority Group Membership of Demonstration Participants 


Percent of Participants 


Priority Group in Priority Group 
Worked less than 3 months out of previous 6 months 33% 
before work registration. 
Received food stamps in 12 of past 24 months. 30% 
Homeless 12% 
Age of 18 and 24 without a high school diploma. 6% 
Primary language other than English. 5% 


Source: E&T/JOBS Conformance Demonstration Project: Implementation and Process Report, Tables 4 
and 5, pp. 30-31. 


Overall, 71 percent of demonstration participants were members of at least one 
priority group. Although, we do not know whether service to these groups increased 
due to the demonstration, it does appear that the demonstration met its goal of serving 


these priority groups. 


Sanctioning 
Hawaii’s impact evaluation reported that only a negligible number of participants 
were sanctioned for failure to respond to the initial call-in in either the demonstration or 
comparison sites. The process and implementation report, however, reported 
substantial numbers of notices of adverse action, both for failure to respond to call-in 
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and for failure to comply with participation requirements after enrollment, as shown in 
Table B-3. The overall sanctioning rate was slightly lower in the demonstration site, 
where 5 percent of all mandatory work registrants were sanctioned, than in the 
comparison site, where 6 percent were sanctioned. This small difference, however, 
masks large underlying differences. The non-compliance rate was lower in the 
demonstration site—8.5 percent of work registrants failed to respond to call-in or 
comply with participation requirements in the demonstration, as compared to 20 percent 
in the comparison site. This large difference in noncompliance was nearly offset by an 
offsetting difference in sanction rate among those in noncompliance—S59 percent in the 
demonstration site as compared to only 31 percent in the comparison site. Thus, while 
the demonstration was more likely to sanction individuals who failed to comply with 
participation requirements, the overall sanction rate among all work registrants was 


slightly lower, primarily because the compliance rate was higher. 


Table B-3 
Sanctioning 


Demonstration | Comparison 


Total number of work registrants wa) 5,792 
Failed to respond to call-in or refused to 5.9% 13.3% 
enroll (%) 

Failed to comply with participation 2.6% 6.7% 


requirements after enrollment 

Sanctions imposed 
Percent of work registrants 5.0% 6.2% 
Percent of failed to respond or comply 58.8% 31.0% 


Source: Hawaii Process and Implementation Report, Table 7, p. ane 


Services 
Types of Services 
The demonstration broadened the range of services offered, to include, for 
example, adult education, post secondary education, and vocational training programs. 


The state’s impact evaluation used multiple regression models to examine the influence 
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of the demonstration on the services received by participants. The results are shown in 
Table B-4, which presents the average monthly percentage of demonstration 
participants receiving each service (both during and before the demonstration) and the 
estimated effect of the demonstration. 


Table B-4 


Average Monthly Participation in Specific Services 


Demonstration Site 


Before ~—dDuring = Regression-Adjusted 
Demonstration Demonstration Change Demonstration Effect 
Basic education 16% 23% +7 -5 
Individual job search 77% 19% -58 -40 
Job search skills training 4% 23% mpd ae 
Vocational training 2% pay 25 120 
Work experience 2% 13% Hil. +17 


Source: Hawaii Food Stamp Employment and Training/JOBS Conformance Demonstration: Impact Evaluation Final 
Report, Tables B-1 to B-5, pp. B-1 to B-S. 


The estimated demonstration effects (shown in the far right column of the table) 
are based on the regression models and differ from the change from before the 
demonstration to the demonstration period (as calculated from the first two columns). 
These differences occur because the multiple regression models adjusted for differences 
between the demonstration and comparison sites in the baseline period and for the 
temporal change in the comparison site, as well as for variations in participant 


characteristics and local economic conditions. 


The results show that the demonstration led to a substantial decline in the percent 
receiving individual job search and increases in the percentages receiving vocational 
training and work experience. There was also a modest decline in the percent receiving 
basic education due to the demonstration. The effect on job search skills training was 
not statistically significant. The increases in vocational training and work experience 
are large relative to the decline in the participation rate discussed above. Therefore, 
not only did the demonstration increase the percentage of participants receiving these 
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services; it actually increased the percentage of all called-in work registrants who 


received these more intensive services, although by relatively small amounts. 


Intensity 

The state’s impact evaluation examined the effects of the demonstration on the 
average length of the different service components for individuals who completed the 
component. Overall, they found that the demonstration decreased the length of basic 
education by 1.6 months (a 34 percent reduction) and increased the length of job search 
skills training by 1 month (a 63 percent increase) for completers of each service. The 
effects on the length of other training components were not statistically significant, 
primarily because the samples of completers in either the demonstration or comparison 
site were too small for reliable analysis. Because these effects were estimated for 
completers, we do not know the effects on the length of services received by all 
participants in a component. Further, the state did not provide information on the 
effect of the demonstration on the overall length of services, which depends on the mix 
of services as well as the length of services for both completers and noncompleters. 


Completion Rates 

Hawaii was able to measure completion rates for participants in various 
components. The only significant finding was that the demonstration reduced 
completion rates for those in independent job search about 33 percentage points, a 
substantial reduction. For several of the other service components the sample of 
recipients was too small for reliable analysis, so we cannot be sure that there was no 
impact. Using data provided in the report, we calculated a simple estimate of the 
change in the overall completion rate and concluded that the demonstration may have 
reduced overall completion rates by about 11 percentage points. The state’s impact 
report suggested that the decline in completion rates occurred because the state 
introduced a work requirement for training participants in the demonstration sites late 
in the research period. This requirement may have induced training participants to 


drop out of training. 


Outcomes 
Employment Rates 
Hawaii’s demonstration program seemed to have little effect on labor market 
outcomes. Before the demonstration, 48 percent of participants were placed in 
employment, while during the demonstration, 44 percent entered employment. 
However, the comparison site had a similar reduction in employment rates during the 
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same period. Even after adjusting for demographic and background effects there was 
no significant relation between being in the demonstration and entering employment. 


Earnings 
Descriptive statistics show that earnings fell in the demonstration site from the 
period before the demonstration period. However, earnings also fell in the comparison 
site at the same time. When adjusting for background and other effects, the 
demonstration was not significantly related to earnings. 


Only slightly more than half the participants employed in the first quarter after 
program exit were employed six months later (based on UI wage records) in both the 
comparison and demonstration sites. Regression results showed that there was a small 
reduction in the probability of being retained (12 percentage points) for demonstration 
participants who were employed right after they left the program. The state’s impact 
report, however, cautioned that the results of this analysis might be unreliable because 
about 7 percent of the sample had incomplete demographic data and was excluded from 
the analysis. 


Overall, it appears that the Hawaii demonstration had a negligible influence on 


labor market outcomes. 


Food Stamp Receipt 
Hawaii’s impact evaluation did not examine the influence of the demonstration on 
Food Stamp receipt. 


Costs 

As shown in Table B-5, Annual spending in the demonstration site increased from 
$194,672 before the demonstration to $911,528 during the demonstration, an increase 
of 367 percent. In contrast, costs increased in the comparison site by only 110 percent. 
Per participant costs rose nearly 11-fold, from $161 to $1,733 in the demonstration, but 
only doubled in the comparison site. Hawaii spent more per participant than any other 


state on its conformance demonstration. 
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Table B-5 
Costs in the Demonstration and Comparison Site 


(Annualized) 
Demonstration Comparison 

Aggregate Costs 

Before demonstration $194,672 $111,327 

During demonstration $911,528 $233,439 

Change ($) $716,856 Sle oel ee 

Change (%) 368 % 111% 
Costs Per Participant 

Before demonstration $161 $126 

During demonstration $157.33 $263 

Change ($) $E572 $137 

Change (%) 976% 109% 


Source: Hawaii Cost Evaluation, Appendix A, pp. A-1 to A-7. 


Hawaii also provided data on cost per participant month, which rose from $142 to 
$1,007, a smaller increase than costs per participant. This is largely due to the fact that 
people were receiving longer services. Costs per participant month in the comparison 
site also rose, but by a lesser amount: from $142 to $243. 


Table B-6 provides information on the distribution of costs. Administration costs 
were considerably higher in the demonstration site, both per participant and as a 
percentage of total costs. Hawaii was the only state where support costs were lower in 
the demonstration site. In the demonstration site, support costs were lower on both a 
per participant and a percentage of total costs basis. Direct delivery costs were 
substantially higher on a per participant basis and slightly higher as a percentage of 


total cost. 


Overall, Hawaii spent more per participant on administration and direct delivery 


than any other state, while spending much less on support services. 
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Table B-6 
Distribution of Costs in Demonstration and Comparison Sites 
(During Demonstration Period January 1994 to June 1995) 


Demonstration Comparison 
Percentage Distribution 
Administration 20% 6% 
~ Direct delivery 78 % 75% 
Supportive services 2% 7 19% 
Per Participant 
Administration $352 $17 
Direct delivery $1,355 $198 
Supportive services $27 $49 


Source: Hawaii Cost Evaluation, Appendix A, pp. A-1 to A-7. 


SUMMARY 


The demonstration project in Hawaii, as in other states, attempted to provide 
more intensive services to a more selected group of priority participants. The impacts 
of the demonstration included: 


e An increase in work registration because of broader work registration 
requirements. 


e A decline in participation rates as services were focused on fewer 
individuals. 


e An increase in the provision of vocational training and work experience 
to participants and a reduction in individual job search among 
participants. The increases in vocational training and work experience 
were large enough relative to the decline in the participation rate so that 
the percentage of work registrants receiving these services increased. 


e No effect on employment or earnings after participation. 


e A substantial increase in costs, aggregate, as well as per participant and 
per participant month. Unlike other states the Hawaii demonstration 
spent less on support costs per participant than did the comparison site. 
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APPENDIX C 
THE EVALUATION OF THE MISSOURI JET 
DEMONSTRATION PROJECT 


THE EVALUATION OF THE MISSOURI JET 
DEMONSTRATION PROJECT 


OVERVIEW 


The JET Program (“JOBS-Employment and Training Demonstration”) is 
Missouri’s Food Stamp Employment and Training/JOBS conformance demonstration. 
Program goals included: 


e Realizing increased efficiency and reduced duplication of effort in 
program administration and operations. 


e Strengthening coordination with JTPA and other state and local 
education and training providers to increase the range and intensity of 
service options available to participants. 


e Improving the match between E&T services and participant interests and 
needs through a thorough client assessment followed by individual 
employability planning, necessary support services, and ongoing case 
management. 


e Encouraging voluntary participation in relatively high intensity and 
long-term activities. 


e Increasing targeting to the least job-ready portion of the Food Stamp 
work registrant population and those most likely to be long-term 
recipients. 


SPR’s process and implementation report provides additional information 
concerning the implementation of the demonstration project (including its goals and 
objectives, its history and evolution, program costs, and other features). This appendix 
summarizes the state’s impact and cost evaluations, including an overview of the 
evaluation design and a discussion of the estimated impacts on participation patterns, 


services, outcomes, and costs. 


EVALUATION DESIGN 

The state chose to use a comparison-site design for its evaluation. The 
demonstration was operated in 8 counties. Outcomes in these counties were then 
compared to outcomes in the remaining (nondemonstration) counties in the state. The 
validity of this design hinges on whether or not there are differences between the 
demonstration and comparison counties, other than the demonstration itself, that might 
lead to differences in outcomes. The state reported that there were only minor 
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differences in education, gender, age, and family size between participants in the 
demonstration and comparison counties. They did identify a substantial racial/ethnic 
difference between the two groups of counties: participants in the demonstration 
counties were approximately 75 percent white, while participants in the comparison 
counties, were only about 50 percent white.! To address this and other potential 
demographic differences, Missouri used a multivariate model to examine differences in 
average earnings Outcomes between the demonstration and comparison sites while 
holding constant race and other demographic variables. Many of the other results 
discussed below, however, are based on simple differences between the demonstration 
and comparison counties. These results could easily be affected by differences in the 
characteristics of participants. 


The major difference between the demonstration and comparison counties is, 
however, in local economic conditions. Seven of the eight demonstration counties are 
in the southeast area of the state, known as the bootheel, which is among the most 
economically disadvantaged areas in the state. Of the eight demonstration counties, 6 
had unemployment rates higher than the state as a whole. These 6 counties contained 
about 2/3 of the work registrants in all demonstration counties. Thus, differences in 
outcomes between the demonstration and comparison sites are likely to be affected by 
economic differences. 


To adjust for these economic conditions, Missouri chose to include in their 
multivariate models a single economic variable: a dummy variable that identified 
counties in the quartile with the lowest employment rates in the state. Unfortunately, 
this economic variable was highly correlated with the dummy variable identifying the 
demonstration counties. As a result, the multivariate analyses reported in the state’s 
final impact report lacked substantial statistical power. This lack of statistical power is 
primarily the result of the state's choice of demonstration counties; given that choice, 
there was little that they could do to effectively control for economic differences during 
the analysis. For this reason, in addition to reporting the state’s final analyses, we also 
report some preliminary analyses that did not control for economic differences. The 
reader should keep in mind, however, that some of the differences detected in these 
preliminary models might be due to economic differences rather than to the 
demonstration itself. 


! Missouri Department of Social Services, Jet Impact Analysis, June 1997, p.3. 
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Another factor that affects the interpretation of the impact results is the large 
difference in participation rates between the demonstration and the comparison 
counties. As discussed below, the demonstration served fewer than 2 percent of 
mandatory work registrants while the comparison counties served over 20 percent. 
Thus, the demonstration participants used in the state’s impact analysis are a small and 
very selected group. Differences between the demonstration and comparison counties 
easily could result from the differential targeting of services in the demonstration rather 


than to differences in the demonstration services themselves. 


Finally, during the demonstration period, the state of Missouri redesigned the 
service components available to the non-demonstration participants so that the E&T 
program in the comparison counties became somewhat more similar to the 
demonstration than had previously been the case. For example, all comparison counties 
also began to offer job training, work experience and job entry experience to E&T 
participants, services that had previously been available only in some counties, through 
E&T contracts with JTPA administrative entities. Nonetheless, substantial differences 
remained between the demonstration and the remainder of the state: the demonstration 
offered more supportive services, initial assessment, case management, and educational 
training. Consequently, the impacts of the demonstration need to be understood as 
resulting from these remaining differences rather than from the full demonstration 


design. 


IMPACT FINDINGS 

Participation Patterns 

Work Registrants 

By implementing the JOBS program’s more inclusive work registration 
requirements, the demonstration was expected to increase the number of mandatory 
work registrants. The state’s impact report did not examine impacts on work 
registration. However, its process report provided some relevant information. From 
the year before the demonstration began until a year after the initiation of the 
demonstration, the number of work registrants in the demonstration counties decreased 
by about 13 percent. According to the process report, this decline may have been the 
result of the overall improvement in the economy during that time period. Indeed, a 
similar decline (about 11 percent) occurred statewide. Thus, there is no evidence that 


the demonstration increased work registration. 
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Participation Rate 

Because the demonstration both targeted services to priority groups and was 
intended to provide more intensive services to those who participated, it was expected 
that participation rates would decline. Although Missouri did not provide information 
on the total number of work registrants and participants for the entire demonstration 
period, it did provide such information for the first year of the demonstration. As 
shown in Table C-1, the overall participation rate among mandatory work registrants 
was markedly lower in the demonstration than in the comparison counties during the 
first year of the demonstration. Only 1.2 percent of work registrants received services 
from the demonstration while over 21 percent of work registrants participated in the 
comparison counties’ E&T program. Clearly, the demonstration succeeded in focusing 


services on relatively few participants. 


Table C-1 
Participation in Demonstration and Comparison Counties 
(October 1994 to September 1995) 


Demonstration Comparison 
Mandatory work registrants 10,168 54,383 
Participants 123 11,276 
Participation rate 1.2% 21% 


Source: Missouri Department of Social Services, Addendum to the Process Evaluation of 
JOBS-Employment and Training Demonstration Program, April 1996, p. 13. 


Volunteers 
The Missouri demonstration succeeded in targeting services to volunteers. Fifty- 
six percent of participants in the demonstration were exempt work registrants, the 
highest percentage of volunteers reported by any of the state demonstrations, while 
there were virtually no volunteers in the comparison counties. Because of the low 
overall participation rate, however, only 68 volunteers participated in the 


demonstration. 


Priority Groups 
Priority groups targeted by Missouri included: 
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e Former JOBS participants who had lost their AFDC eligibility. 


e Individuals who were high school dropouts, had little or no work 
experience, and/or were long-term Food Stamps recipients. 


e Displaced homemakers or former AFDC recipients. 


e Absent parents with pending child support claims. 


For the most part, the state did not provide information on precisely these priority 
groups. However, the state’s process report identified some substantial differences in 
the demographic characteristics of E&T participants in the demonstration and 
comparison counties, as summarized in Table C-2. 


Table C-2 
Participant Demographic Characteristics in Demonstration and 
Comparison Counties 
(October 1994 to September 1995) 


Demonstration Comparison 
Female 82% 53% 
White 80% 50% 
High school dropout 29% 33% 
Under 4 months of work experience 53% 21% 


Source: Missouri Department of Social Services, Addendum to the Process Evaluation of 
JOBS-Employment and Training Demonstration Program, April 1996, Attachments C and E. 


Some of the demographic differences between the demonstration and comparison 
sites were consistent with expanded service to the demonstrations priority groups; 


others were not. 


Consistent with its targeting criteria, the demonstration served a greater 
percentage of females and individuals with little work experience than did the 
comparison counties. In contrast, the percentage of participants who were high school 
dropouts was actually slightly lower in the demonstration site than in nondemonstration 
counties, so that the demonstration did not succeed in giving priority to dropouts. 
There was also a marked difference in the ethnic makeup of participants: 80 percent of 
participants in the demonstration were white, while only 50 percent were white in the 
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remainder of the state. The writers of the state’s process report thought that this very 
large difference could not be explained by demographic differences in the area—the 
percentage of the population that is white is only slightly higher in the demonstration 
counties than in the comparison counties (93 percent as compared to 87 percent). 


On balance, the demonstration project likely achieved most of its targeting goals. 


Sanctioning 


Missouri did not implement sanctions until late in the demonstration. 


SERVICES RECEIVED 

Types of Services 

The demonstration was intended to provide a broader array of services to E&T 
participants. To examine whether the demonstration was successful in providing 
enhanced services, the state provided some descriptive statistics on the distribution of 
services for the demonstration and comparison counties, as summarized in Table C-3. 


Table C-3 
Distribution of Services in Demonstration and Comparison Counties 
(October 1994 to September 1995) 


_Demonstration __Comparison__ 
Assessment 22% 0% 
Job readiness/job search 13% 32% 
H.S./GED/ABE 14% 0% 
Postsecondary education 12% 0% 
Job skills training 11% 17% 
Work experience 4% 8% 
On-the-job training 0% 2% 


Note: percentages are based on the total number of services (and sum to 100 percent), not 
on the number of participants. Because some participants received multiple services, a 
greater percentage of participants received each service 


Source: Missouri Department of Social Services, Addendum to the Process Evaluation of 
JOBS-Employment and Training Demonstration Program, April 1996, p. 17. 
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The major differences between the demonstration and comparison counties are 
consistent with the overall goals of the demonstration. The demonstration was more 
likely to provide assessment and educational training than the comparison counties. 
The comparison counties were more likely to provide job-search-related activities and 
occupational related-training, such as job skills training and work experience. In 
addition the demonstration tended to provide more services per participant. 


Intensity of Service 


Missouri did not provide information on the intensity of services offered. 


Completion Rates 


Missouri did not provide any quantitative information on the completion rates. 


Outcomes 
Employment Rates 

Missouri’s final impact report did not examine employment rates. However, the 
state provided descriptive statistics that show that 86 percent of 451 participants in the 
demonstration were employed at termination.? In contrast, only 65 percent were 
employed at termination in the comparison counties. Although this simple comparison 
does not account for the economic differences between the demonstration and 
comparison counties, one would have expected Jower employment rates in the 
demonstration counties because of the poorer economic conditions discussed above. 
There are, however, other factors that may have influenced this difference. The 
demonstration had a higher percentage of participants who were more motivated 
(volunteers), females, and white. These factors might explain some of the employment 
rate difference. However, not all these factors necessarily work in the same direction 
(e.g., females generally have lower employment rates than males while whites typically 
have higher employment rates than nonwhites) and the employment difference is fairly 
large relative to the demographic differences. Therefore, there is some evidence that 
the demonstration had a positive effect on employment rates for the relatively small 
number of individuals who received demonstration services. Because the participation 
rate was so low, however, there can be little confidence that their result also applies to 


all work registrants 


2 These data cover the period from October 1994 to March 1996. 
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Earnings 
In its final impact report, Missouri examined the effects of the demonstration on 
two measures of earnings: average weekly earnings derived from UI wage records and 
earned income as reported in the Food Stamp files. As discussed above, the statistical 
power of these analyses was quite small because a measure of local economic 
conditions included in the model was highly correlated with the dummy variable 
identifying the demonstration counties. As a result, Missouri not find a significant 


effect of the demonstration on either outcome. 


A previous draft of the impact report examined the influence of the demonstration 
on reported income without trying to control for regional economic differences. ° This 
analysis indicated that the demonstration had a significant positive effect on reported 
income, increasing income by $100 per week. Because the demonstration was 
conducted in relatively economically depressed areas, one would expect that adjusting 
for economic conditions would increase the estimated effect. The models also did not 
adjust for gender and ethnicity, two factors that are known to be quite different between 
the demonstration and comparison counties. Although these differences might explain 
some of the difference in earnings, their influences likely offset each other: the higher 
percentage of females in the demonstration would tend to lead to lower earnings while 
the higher percentage of whites would tend to lead to higher earnings. Overall, the 
result indicates that the demonstration likely increased earnings among participants. 


CosTs 
As shown in Table C-4, Missouri spent four times as much per participant in the 
demonstration as in the comparison site: $1,274 in the demonstration site as compared 


to $317 in the comparison site. 


Both support costs and other costs per participant were substantially higher in the 
demonstration, although there was also a shift in the distribution of costs towards 
support costs. The increase in costs per participant is consistent with the decline in the 
participation rate discussed above. As discussed above, the participation rate in the 
demonstration was only 1 percent while the participation rate was 21 percent in the 


3 Missouri Department of Social Services, JET Impact Analysis, Draft, March 1997. The analyses 
reported in this version of the report did, however, control for family size, age, and education. 
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comparison site. Thus, a relatively small number of individuals were served in the 
demonstration at a relatively high cost per participant. 


Table C-4 
Costs in the Demonstration and Comparison Counties 
(October 1994 to March 1996) 


Demonstration Comparison 
Total cost per participant $1,274 $317 
Support cost per participant $502 $73 
Other cost per participant $772 $244 
Percent spent on support costs 40% 23% 
Source: Missouri Department of Social Services, JET Cost Evaluation, December 1996, 
Tableria pet: 

SUMMARY 


The Missouri demonstration strengthened work registration requirements, 
encouraged participation by volunteers, targeted priority groups, and offered 
participants a wider range of services. Key impacts of the demonstration include: 


e Little change in the number of work registrants. 


e A dramatic decline in the participation rate, from 21 percent to 1 
percent. 


e Increased service to females, whites, and to individuals with limited 
work experience. 


e Participation in educational services by about a quarter of participants 
and declines in the percentage of participants receiving job readiness/job 
search, job skills training and job entry. 


e An increase in employment upon exit from the E&T program and an 
increase in reported income. 


e A quadrupling of cost per participant, with the largest increase 
occurring for support costs, which amounted to 40 percent of all costs. 
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APPENDIX D 
THE EVALUATION OF THE SOUTH DAKOTA FAMILY 
INDEPENDENCE FOOD STAMP EMPLOYMENT AND 
TRAINING/JOBS CONFORMANCE DEMONSTRATION 


THE EVALUATION OF THE SOUTH DAKOTA FAMILY 
INDEPENDENCE FOOD STAMP EMPLOYMENT AND 
TRAINING/JOBS CONFORMANCE DEMONSTRATION 


OVERVIEW 

South Dakota’s Family Independence Food Stamp Employment and 
Training/JOBS Conformance Demonstration was implemented statewide in all 20 
counties in which E&T services are offered. As stated by the South Dakota 
Department of Social Services (DSS), the objectives of the demonstration were to 
improve the operations, effectiveness, and efficiency of the Food Stamp E&T program 
by: 


e Improving the efficiency of program operations, by conforming work 
registration and sanctioning rules for JOBS and E&T. 


e Making the services provided to program participants more 
comprehensive and better designed to promote self-sufficiency for 
households receiving Food Stamps. 


e Encouraging participants to follow through on their commitments by 
increasing staff follow-up and strengthening the sanctioning process. 


The demonstration was implemented in all counties in the state that had 
operational E&T programs. 


SPR’s process and implementation report provides additional information 
concerning the implementation of the demonstration project (including its goals and 
objectives, its history and evolution, program costs, and other features). This appendix 
summarizes the state’s impact and cost evaluations, including an overview of the 
evaluation design and a discussion of the estimated impacts on participation patterns, 


services, outcomes, and costs. 


EVALUATION DESIGN 
Because the demonstration operated statewide, the state’s evaluation design 


compared E&T before and during the conformance demonstration (i.e., a pre/post 
analysis). The pre/post design relies on the assumption that the introduction of the 
demonstration is the only change between the predemonstration and demonstration 


periods that materially affects the outcomes and other dependent variables being 
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studied. The major factors that might have changed between these two periods are 
economic conditions, which can be expected to affect outcomes, and the characteristics 


of E&T participants. 


To address the threats to the validity of the evaluation posed by these potential 
confounding factors, South Dakota’s evaluation employed multivariate models that 
controlled for differences in participant characteristics and for temporal and geographic 
differences in economic conditions. Participant characteristics used in these regression 
models included, for example, gender, race, and age. Local economic conditions 
included the number of employed and unemployed in the county. These multiple 
regression models were used to examine two outcomes—employment and earnings. 
The outcomes were examined only for two subgroups of participants: individuals 
receiving job search and individuals receiving job readiness services. 


In addition, the state’s evaluation included a variety of descriptive analyses that 
allow a simple pre/post comparison or that examined demonstration participants in 
detail. 


Several limitations should be kept in mind when assessing the estimated impacts 
of the demonstration: 


© Much of what is discussed below is based on the descriptive analyses. 
The reader should keep in mind that these descriptive analyses are 
subject to confounding influences by temporal changes in participant 
characteristics and economic conditions. This qualification applies to all 
the impacts discussed below except for the impacts on employment and 
earnings. 


e The impact models estimated by the state for employment and earnings 
were conducted only for the subgroups receiving job search or job 
readiness services. They do not include E&T participants receiving new 
services that were introduced by the demonstration. Therefore, the 
estimated impacts do not reflect the benefits of the expanded services 
available through the demonstration. 


e The impact evaluation did not report the magnitude of the estimated 
impacts from the regression models. It also did not report the statistical 
significance of the findings. We assume that all reported results are 
significant, both statistically and in terms of their absolute size. 
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IMPACT FINDINGS 
Participation Patterns 
Work Registrants 


As a result of applying more inclusive JOBS work registration requirements, the 
demonstration was expected to increase the number of mandatory work registrants. As 
expected, South Dakota had an increase in the number of mandatory work registrants 
over time. As reported in the state’s impact study!, the number of mandatory work 
registrants rose from an average of 4,698 per year before the demonstration to 6,983 
during the demonstration, an increase of 49 percent. Over the same time period, the 
total number of Food Stamp recipients in the state declined slightly. Therefore, we can 
be confident that the demonstration increased work registration among recipients. 


Most of the increase in work registration appears to have resulted from 
eliminating the exemption for caretakers of young children—South Dakota required 
work registration by caretakers of children as young as one year old. The percentage 
of work registrants who were female increased from 44 percent to 57 percent and the 
percentage who were parents with children increased from 48 percent to 63 percent. 


Participation Rates 


Because the demonstration both targeted services to priority groups and was 
intended to provide more comprehensive services to those who participated, it was 
expected that participation rates would decline. The results presented in the state’s 
impact report show that the participation rate fell by half after introduction of the 
demonstration. The participation rate before the demonstration was 31 percent (4,436 
participants out of 14,095 work registrants). For the demonstration as a whole, it was 
15 percent (2,554 participants out of 17,457 work registrants). The number of 
participants declined from an annual average of 1,479 per year before the 
demonstration to 1,022 during the demonstration. 


Volunteers 


South Dakota did not encourage participation by volunteers in the demonstration. 
As a result, only 9 exempt volunteers received any E&T services during the 


demonstration. 


1 The state’s impact report uses a different methodology in computing the number of participants, 
so the numbers presented here cannot be directly compared to the state’s process report. 
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Priority Groups 

South Dakota identified several priority groups for services, including heads of 
household under age 24, individuals under age 24 with limited work experience, long- 
term Food Stamp recipients, and volunteers. As intended, the demonstration succeeded 
in focusing resources somewhat on younger work registrants. As shown in Table D-1, 
the percentage of participants age 25 or under increased from 30 percent before the 
demonstration to 41 percent during the demonstration. There was also a marked 
increase in the percentage of participants who were female, from 41 percent to 61 
percent. Similarly, there was an increase in the percentage of parents with children, 
from 46 percent to 66 percent. These changes in gender and family composition among 
participants largely mirrored similar changes among all work registrants, so the 
changes probably resulted primarily from changes in work registration requirements. 
For example, caretakers of children between the ages of 1 and 6 were no longer 
exempted from work registration, a change that may have led to the increased relative 
service to females, young adults, and parents with children. Among participants, the 
percentage of long-term food stamp recipients (a priority group defined as being a food 
stamp recipient 36 out of 60 months prior to the initial application) increased from 28 
percent to 32 percent. Other demographic characteristics showed smaller changes. 


Table D-1 
Characteristics of Work Registrants and Participants 





Work Registrants Participants 
Before Before 

Demonstration Demonstration Demonstration Demonstration 
Age 25 or under 33% 35% 30% 41% 
Female 43% 57% 41% 61% 
At least one child in 50% 65% 46% 66 % 
household 
Food stamp recipient 23% 25% 28% 32% 
in 36 of 60 months 
White 10% 71% 71% 16% 
Source: Business Research Bureau, South Dakota’s Food Stamp Employment & Training/Job Opportunities and 
Basic Skills Conformance Demonstration: A Report of the Findings of the Impact Evaluation Study, September ] 


1996, Tables 3 and 4, pp. 54-55. 
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It should be noted that, although service to these groups increased as a percentage 
of all participants, because of the sharp drop in the participation rate, work registrants 
in these groups were actually less likely to receive services during the demonstration 
than previously. For example, before the demonstration about 30 percent of female 
work registrants received some E&T services. During the demonstration, only about 
18 percent of female work registrants received services. Thus, the focus of the 
demonstration on providing intensive services to a few individuals overwhelmed the 


more directed targeting of services on priority groups. 


The state also provided descriptive information about service to priority groups 
during the demonstration: about 42 percent of all participants who received services 
beyond job search and job search training (such as education, classroom training, and 
on-the-job training) were members of one of the priority groups; nearly 20 percent 


were under age 24 with little work experience.” 


Overall, the data show that South Dakota was able to target resources as intended 


during the demonstration. 


Sanctioning 
In South Dakota, notices of adverse action were sent to 46 percent of work 
registrants in the first full year of the demonstration as compared to 49 percent in the 
year before the demonstration, a decline of 3 percentage points. The absolute number 
of notices of adverse action then increased dramatically in the second year of the 
demonstration, but we don’t know whether there was a corresponding increase in the 
number of work registrants. Therefore, the overall effect of the demonstration on 


sanctioning in South Dakota is unclear. 


Services 
Types of Services 
The demonstration offered a broader range of services than was previously the 
case. Enhanced services included secondary education, skills training, on-the-job 
training, work experience, and community service. Of the 2,554 participants who 
received any service, 21 percent (544) received one of these enhanced services. As 


2 Business Research Bureau, South Dakota’s Food Stamp Employment & Training/Job 
Opportunities and Basic Skills Conformance Demonstration: A Report of the Findings of the Impact 
Evaluation Study, September 1996, Table 18, p. 69. 


D-5 Social Policy Research Associates 


shown in Table D-2, the most frequently used enhanced service was educational 
services, which was received by 14 percent of participants. The remaining enhanced 
services were received by a relatively small percentage of participants. Overall, about 
20 percent of participants received enhanced services introduced by the demonstration. 
A smaller percentage of participants received job search and job readiness services 
during the demonstration than previously, a result that was confirmed by multivariate 
regression analyses. The decline in job search was small—over 80 percent of 
participants received job search, as they had before the demonstration. Thus, the 
demonstration generated only modest changes in service mix, possibly because the 
increased funding for the demonstration was spread across the entire E&T program in 


the state. 


Table D-2 
Effects of Demonstration on Service Receipt 
(Percent of Participants) 


Before Difference 


Demonstration Demonstration (Percentage Points) 


Job readiness 21% 30% -9 
Job search 84% 88% -4 
Educational services 14% 0% +14 
Vocational skills training 2% 0% se 
On-the-job training 2% 0% too 
Work experience 1% 0% sth 
Community service 3% 0% +3 


Source: Calculated from South Dakota’s Impact Report, Tables 1, 18, 28, and 31; pp. 52, 69, 79, and 
82. 


Intensity 
The length of time in training typically ranged from 1 to 3 months. For the 
traditional programs of job readiness and job search, the average time spent was 1.2 


and 2.4 months respectively (there was little change from the predemonstration period). 


The other service components lasted about a month longer, in the range of over 3 
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months. Because the demonstration increased participation in the longer components, 
there was an overall increase in the intensity of training services. 


Completion Rates 


No information was given on training completion rates for South Dakota. 


Outcomes 
Employment Rates 
The South Dakota evaluation used multivariate models to assess the influence of 
the demonstration on employment for two subgroups of participants: those receiving job 
readiness training and those receiving job search. These models estimated the impact 
of the demonstration while controlling for demographic characteristics and economic 
conditions. 


The multivariate models for employment showed positive demonstration effects 
on employment at both 6 and 12 months after referral to E&T for both the job readiness 
and job search groups.? We cannot, however, infer from these results that the overall 
impacts on employment of participants were positive, although the results are 
suggestive since over 80 percent of participants received job search and, thus, were 


included in the models. 


Earnings 

The South Dakota impact report examined impacts on earnings in two different 
ways. First, it provided descriptive data on earnings for all work registrants, not just 
participants, both during and before the demonstration. These data allow an assessment 
of the overall effects of employment combining both the effects on participants and the 
effects on nonparticipants who would have received services if the demonstration had 
not reduced the participation rate. The descriptive data indicate that, earnings 6 months 
after referral to E&T was a few dollars higher during the demonstration than before and 
that earnings 12 months after referral was about $100 higher after the demonstration 


than before. 


Second, it provided multivariate estimates of the impacts on earnings (for both 
those employed and those not employed) for the job search and job readiness 
subgroups. These results were mixed: there were positive effects on earnings 6 and 12 


3 South Dakota’s Impact Report, Table 64A, p. 115. No numerical estimates were presented. 
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months after referral for the job readiness group, but negative impacts were found for 
the group receiving job search services. Since the job search group is nearly four times 
the size of the job readiness group, the overall demonstration impact is likely to have 


been negative. 


Because these multivariate analyses control for demographic and economic 
conditions, we consider them to be more reliable than the simple descriptive analysis. 
Therefore, the overall conclusion is that there were no positive effects on earnings, 


despite the increase in employment 


Food Stamp Receipt 


Descriptive data for all work registrants show that the average benefit amount was 
higher during the demonstration than previously, by $18, 6 months after referral to 
E&T and by $9, 12 months after referral, despite lower benefits (by $24) in the referral 
month. Further analysis showed that these differences were concentrated among female 
work registrants. Male work registrants tended to have slightly lower benefits during 
the demonstration. These data suggest that, if anything, the demonstration may have 
increased Food Stamps benefits overall, possibly because services were provided to a 


smaller proportion of all work registrants. 


CosTs 


As shown in table D-3 annual spending for the E&T program increased from 
under $500,000 per year before the demonstration to $700,000 per year during the 
demonstration, a 43 percent increase. Costs per individual assessed increased by an 
even greater percentage, 121 percent, due to the decline in the participation rate. Thus, 
costs per individual assessed increased from $141 to $311.4 


J 


4 Costs per participant are higher because many individuals were assessed but did not receive 
subsequent services. Participant counts for the same time periods covered by the cost data were not 
provided. 
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Table D-3 
Costs During and Before the Demonstration 


During Before 
Demonstration Demonstration Change (%) 
Aggregate Costs (Annualized) $699 ,947 $488,463 43% 
Cost per individual assessed $311 | $141 121% 


South Dakota also reported on leveraged funds, that is funds from other sources 
used to serve demonstration participants. However, cost information on child care, 
financial aid, and postsecondary ‘education could not be reported. The remaining 
leveraged funds, from basic education programs (e.g., ABE and GED) and JTPA- 
funded services were estimated at $257,118 for the three-year demonstration period, 
serving about 400 individuals. The reported leveraged funds enabled South Dakota to 
increase resources by devoted to participant services by 12 percent. 


SUMMARY 


The South Dakota demonstration implemented more inclusive work registration 
requirements, targeted some priority groups, and offered a broader array of services to 
E&T participants. Key impacts of the demonstration include: 


e A substantial increase in the number of work registrants, possibly by as 
much as 4,990. Most of the increase appears to result from eliminating 
the exemption for caretakers of young children. 


e A halving of the participation rate from 31 percent to 15 percent of 
mandatory work registrants. 


e Large increases in the percentages of participants who are young, 
female, or parents with children. 


e Provision of new services, primarily secondary education, to about 20 
percent of participants. 


e A modest increase in the average length of services. 


e Positive effects on employment for those receiving job search or job 
readiness, but little or even negative effects on earnings. 


e An increase of over 100 percent in costs per individual assessed. 


Overall the demonstration appears to have succeeded, although the evidence of 


impacts on employment are weak. 
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APPENDIX E 
THE EVALUATION OF THE TEXAS BOND 
DEMONSTRATION PROJECT 


THE EVALUATION OF THE TEXAS BOND 
DEMONSTRATION PROJECT 


OVERVIEW 


The Texas BOND demonstration was implemented in a single county. As 
described by the Texas Department of Human Services (DHS),! the objectives of the 
demonstration included: 


e Providing expanded and enhanced activities and support services for 
E&T participants, both through services directly funded with E&T 
dollars and services provided by collaborating education and training 
agencies from other funding streams. 


e Improving participation through a stronger sanctioning policy. 


e Targeting resources based upon participant need, rather than operating a 
“one size fits all” E&T program. 


e Providing continuity of services to E&T and JOBS participants who 
experience a change in program eligibility while participating in 
employment and training services. 

e Increasing program efficiency and reducing program costs through 


common administrative processes, support materials, staff training, and 
a single service delivery system. 


e Assisting participants to move toward self-sufficiency. 


A unique feature of Texas demonstration was its two-tiered service design. High 
school graduates with recent work experience were generally provided with job search 
and job readiness activities, which were identical to those provided in the comparison 
site. In the demonstration, these services were supplemented with group case 
management. High school dropouts with 8th grade completion and recent work history 
were typically referred to components that addressed their education or social skills 
deficits, such as adult education or special survival skills training, along with individual 
case management. In the comparison site these individuals received the job search and 
job readiness services. Child care support was available to both groups of individuals 


in the demonstration. 


1 The administration of the Food Stamp Employment and Training Program was transferred to the 
new Texas Workforce Commission, effective June 1, 1996. 
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As part of the demonstration, there was a full consolidation of the E&T and JOBS 
programs in the county. JOBS policies and procedures were applied to participants in 
both programs, staff serving the two client groups were merged, case management and 
service delivery procedures were consolidated, and, with few exceptions, identical 
services were provided to JOBS and E&T participants. Among all the states, the Texas 


demonstration model most closely matched JOBS program goals. 


SPR’s process and implementation report provides additional information 
concerning the implementation of the demonstration project (including its goals and 
objectives, its history and evolution, program costs, and other features). This appendix 
summarizes the state’s impact and cost evaluations, including an overview of the 
evaluation design and a discussion of the estimated impacts on participation patterns, 
services, outcomes, and costs for the period October 1993 to September 1995. FY 
1996 is not covered by the evaluation because of a restructuring of the state’s welfare- 


to-work programs. 


EVALUATION DESIGN 

Texas chose to use a combined comparison site/pre-post design to evaluate the 
impacts of its demonstration. The demonstration was operated in McLennan County, 
while the state selected Smith County as the comparison site. These two counties are 
quite similar economically. Both contain small cities (Waco and Tyler respectively) 


and have similar industrial mixes, unemployment rates, and poverty rates. 


Data provided in the impact report show, however, that there were substantial 
differences between the demonstration and comparison sites in the demographic 


characteristics of E&T participants before implementation of the demonstration, as 


shown in Table E-1. 


Because of these substantial differences, the state’s evaluators were not willing to 
rely on simple comparisons between the demonstration and comparison sites to 
determine the impacts of the demonstration. Instead, the estimate of the demonstration 
impact was based on the difference between the demonstration and comparison sites of 
the change from before to during the demonstration period. For example, if earnings 
increases by $15 in the demonstration site and increases by $5 in the comparison site, 


then the estimate of the impact is $10 (15 - 5). 
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Table E-1 
Demographic Characteristics of E&T Participants in Demonstration and 
Comparison Sites Before Implementation of the Demonstration 


Demonstration Comparison 


Female 40% 47% 
White 27% 35% 
Black 60% 62% 
Hispanic 12% 3% 
Age 30 or under 28% 36% 
One person household 53% 33% 
High school graduate 56% 75% 


Source: Texas Impact Evaluation Final Report, Table 5, p. 14. 


The strength of the combined design is that it adjusts both for preexisting 
differences between the demonstration and comparison site and for trends over time that 
are common to both sites. Thus, the main threat to the validity of the estimates is 
differential changes in participant characteristics or economic conditions. To address 
this possibility, the state’s impact evaluation estimated regression models that controlled 
for a variety of participant demographic characteristics, including household 
composition, gender, age, ethnicity, high school dropout. Selected models also 
included variables for whether previously sanctioned, volunteer, and preprogram 
earnings. Many models also included several economic variables: employment growth, 
income growth, and the unemployment rate. Because there were only one 
demonstration and one comparison county, these county-level economic variables are 
likely to be highly correlated with the variable measuring the demonstration impact. 
This correlation may have seriously reduced the statistical power of the regression 


models. 


During the period of the demonstration there were several statewide changes that 
affect the interpretation of the results. In particular, the E&T program in the 
comparison site experienced some changes that made it more like the demonstration. 
The most important of these changes was a statewide redesign of the job search and job 
readiness components offered to E&T participants that occurred just as the 
demonstration was beginning. As a result of this redesign, E&T participants received 
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expanded and intensified job search and job readiness services, which were similar in 
both the demonstration and comparison sites. It also reduced capacity to serve clients 
in many E&T offices. In addition, transportation allowances available to E&T 
participants were increased to conform transportation payments statewide between E&T 
and JOBS. As a result of these changes in the comparison site, the major service- 
design features that distinguished the demonstration site from the comparison site were 
(1) increased access to a wide variety of additional activities, including initial 
assessment, group OF individual case management, education, vocational skills training 
and work experience and (2) increased access to supportive services, especially child 
care assistance. The estimated impacts of the demonstration should be interpreted as 


those resulting from these remaining differences. 


The remainder of this appendix summarizes the findings of the Texas impact and 
cost reports on work registration, participation rates, participant demographics, 


services, outcomes, and costs. 


IMPACT FINDINGS 

Participation Patterns 

Work Registrants 

The demonstration strengthened work registration rules to include three new 
groups: unemployment compensation recipients, participants in residential substance 
abuse treatment programs, and caretakers for children 3 to 5 years old. The 
demonstration was, therefore, expected to increase the number of work registrants. 
The state’s impact report conducted a simulation of the change in work registration 
rules and found that the number of exemptions decreased by about 8 percent as a result 
of the new rules. Over 80 percent of these new work registrants were affected by the 
elimination of the exemption for caretakers of children aged 3 to 5 years. The decline 
in exemptions suggests a substantial increase in mandatory work registration: about 
25%. Overall, only 2/3 of adult Food Stamps recipients were exempt from E&T 
participation in the demonstration county as compared to over 80 percent in the 


comparison county. 
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Participation Rates 


Because the demonstration was intended to provide more intensive services to 
priority groups, it was expected that the participation rate would decline.2, As shown in 
Table E-2, the participation rate among mandatory registrants in the demonstration 
county fell from 4.7 percent before the demonstration to 2.3 percent in the first year of 
the demonstration and increased to 3.9 percent in the second year. The comparison 
site, however, experienced even larger declines as the number of participants dropped 
dramatically. By the second year of the demonstration, participation rates were the 
same in the demonstration and comparison site even through the participation rate was 
much higher in the comparison site before the demonstration term. As a result, the 
simple difference in means pre-post analysis presented in the table indicates that the 
demonstration actually increased participation among mandatory work registrants. The 
estimated effects, however, are not reasonable in size because they exceed the actual 
participation rate in the demonstration. The increase in the participation rate is 
confirmed by regression models that hold constant background and other economic 
variables, although the effect is somewhat smaller, an increase of about 2 percentage 
points. Texas was unique among demonstration states in finding that the demonstration 
increased participation rates, which was enabled by nearly four-fold increase in 
expenditure in the demonstration site. 


2 Participant was defined in Texas as an individual who had some actual hours beyond assessment 
recorded in an E&T activity. 
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Table E-2 
Participation Rates for Mandatory Registrants 
in the Demonstration and Comparison Sites 


Demonstration Effect 
Demonstration Comparison (percentage points) 


Before demonstration 4.7% 11.8% 

First year of 2.4% 5.9% 

demonstration 

Change from before -2.3 -5.8 +3.5 
Second year of a9 Se, 

demonstration 

Change from before -0.8 -7.9 +7.1 


Source: Texas Impact Evaluation Final Report, Table 7, p. 17. 


Volunteers 

A major difference between the demonstration and comparison sites was that the 
demonstration allowed exempt individuals to participate voluntarily in E&T services. 
Volunteers were very important in demonstration participation patterns, as they made 
up over 40 percent of demonstration participants. Volunteers were more likely to be 
female, white, have a higher education, more children, and have more job experience 
than were those who did not volunteer. Individuals with significant barriers to 
employment were less likely to volunteer as well. This demographic profile of 


volunteers is similar to that of volunteers in other states. 


Participant Characteristics 
The Texas demonstration did not change priority groups and used the same 
procedures as the remainder of the state to determine which work registrants would be 
required to participate. Thus, the only aspects of the demonstration likely to affect 
participant characteristics are the changes in the mandatory work registration 


requirements and the extensive service to volunteers. 


There were some substantial changes in participant characteristics due to the 


demonstration, as shown in Table E-3. There was a sharp increase in the percentages 
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Table E-3 
Characteristics of Demonstration Participants 


Demonstration 
Percent of Participants Effect 
During Before (percentage 
Demonstration Demonstration points) 
Female 69% * 40% AS, 
White 50% 27% m2 
Black 36% 60% -24 
Hispanic 14% 12% sco 
Age 30 or under 55% 28% anOat 
One person household 23% 53% -30 
High school graduate 82% 56% +26 


Source: Texas Impact Evaluation Final Report, Table 5, p. 14. 


of participants who were female or under age 30 and a decline in the percentage in one- 
person households. These changes are consistent with the removal of exemptions for 
those caring for children aged 3 to 6. In addition, there was a marked increase in the 
percentage white, a decline in the percentage black and an increase in the percentage of 
high school graduates. These changes are consistent with the heavy service to 
volunteers. Changes in the comparison site over the same time period were much 


smaller and often in the opposite direction. 


Sanctioning 

Because the demonstration adopted JOBS sanctioning rules, some impact on 
sanctions was expected. Texas had problems with tracking data on sanctions imposed. 
Instead, the evaluators examined sanctions requested by employment workers for 
failure to respond to the initial call-in. During the demonstration, sanctions were 
requested for 43 percent of called-in work registrants. Both descriptive and multiple 
regression analyses indicate that the percentage of called-in work registrants for whom 
sanctions were requested declined by over 15 percentage points due to the 
demonstration; the estimate from the regression analysis is a decline of 17 percentage 


points. 
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Services 

The demonstration was intended to provide participants with expanded and 
enhanced activities and support services, as well as group or individual case 
management. Available services included job skills training, post-secondary education, 


and work experience. 


As shown in Table E-4, the distribution of services received by participants 
changed markedly in the demonstration. The estimated demonstration effects, which 
account for changes in the comparison site as well as the change in the demonstration 
site, show that there was a substantial reduction in the receipt of directed job search and 
a substantial increase in educational services. Most participants in educational services 
received postsecondary education at a community college; others received GED and 
high school education; a few received English as a second language or adult basic 
education. Regression models that controlled for participant characteristics and time 


trends showed similar results. 


Table E-4 
Services Received by Participants in the Demonstration 








Demonstration Site Demonstration 
(Average Monthly Percent) Effect 
During Before (percentage 
Demonstration Demonstration points) 

Directed job search 11% 78% -66 
Job readiness 15% 14% -3 
Vocational training 1% 0% mel 
Education services 74% 9% +65 | 
Work experience 4% 0% +4 


Source: Texas Impact Evaluation Final Report, Tables A-1 to A-5, pp. A-1 to A-5. 


Interestingly, training activities received by demonstration participants differed by 
exemption status. Nearly all volunteers participated in educational activities, whereas 
nonvolunteers were split equally between educational activities and the remaining 
activities (mostly the traditional E&T training programs of directed job search and job 
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readiness). Thus, providing services to volunteers enabled exempt individuals to access 
educational activities. 


Intensity 


There was also an increase in the intensity of training services offered. The 
overall average monthly hours of activities increased by 61 hours from 24 to 86 hours 
per month in the demonstration site as compared to an increase of just 35 hours in the 
comparison site. Most of this change was due to the large amount of vocational and 
educational training provided in the demonstration. There was little change relative to 
the comparison site in monthly hours of directed job search and job readiness, both of 
which increased in intensity and had similar designs in both the demonstration and 
comparison Sites. 


Completion Rates 

Texas provided information on educational outcomes of its participants. Thirteen 
percent of participants who enrolled in GED-oriented components before the 
demonstration actually obtained a certificate. During the demonstration, that figure 
rose to 17 percent. The absolute number of GEDs attained increased by an even 
greater amount because many more participants received GED training. There was also 
a marked increase in the attainment of a postsecondary degree or certificate: the number 
receiving such a degree or certificate increased from none before the demonstration to 
112 over the first two years of the demonstration. Regression analyses that adjusted for 
personal characteristics and other background variables show only modest and not 
Statistically significant impacts of demonstration on receipt of postsecondary degrees or 
certificates. The authors of the impact report explain this apparent inconsistency by 
noting that the participants in the demonstration had more educational and employment 
skills—largely due to the increased numbers of volunteers. It is these background 
characteristics that appear to be the cause of the higher educational outcomes rather the 


demonstration itself. 


Outcomes 
Employment Rates 
Employment at program completion increased substantially in the demonstration 
site: from 48 percent before the demonstration to 61 percent during the demonstration. 
However, somewhat smaller increases also occurred in comparison site. Regression 
models that adjust both for the trend in the comparison site and for demographic 
characteristics and economic conditions indicated that the demonstration did not have a 
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significant influence on employment rates. As with the educational outcomes discussed 
above, other demographic factors such as age, race, and education level played a larger 
role than the demonstration itself in the increase in employment rates. 


Texas also measured longer-term estimates of employment outcomes. Long-term 
employment was defined as earning at least $1,500 for four consecutive quarters after 
termination. The descriptive statistics show that employment rates are higher for the 
demonstration (8 percent before the demonstration and 15 percent during the 
demonstration), but, once again, they are not significant when holding constant 


demographic factors. 


Earnings 
Although employment rates at termination were similar for both the 
demonstration and comparison sites, demonstration participants did appear to have 
slightly higher quarterly earnings at termination. On average, quarterly earnings of 
employed demonstration participants increased by about $375 relative to the increase in 
the comparison site. Regression models, however, again indicated that there was no 


increase after adjusting for demographic and economic differences. 


To conclude, while descriptive statistics show some small gains in employment 
and earnings, most of these gains can be attributed to demographic factors such as age, 
race, and educational level, which in turn may have increased because of the high level 


of service to volunteers. 


Food Stamp Receipt 


Texas did not provide information on Food Stamp receipt. 


Costs 
Total spending in the demonstration site increased from $251,000 per year to an 
average of $920,000 per year during the two years of the demonstration. Total costs in 


the comparison site also increased, but by a much smaller amount. 


Per participant costs in the demonstration were about two to three times what the 
comparison county spent per participant and more than twice what was spent before the 
demonstration program, as shown in Table E-5. Thus, it seems likely that the 


demonstration at least doubled costs per participant. 
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Table E-5 
Costs per Participant in the Demonstration and Comparison Sites 


Demonstration Comparison 


Before demonstration $296 — 
First demonstration year $1,035 $336 
Second demonstration year $1,318 $680 


Source: Calculated from data in Texas Cost Analysis Final Report and 
Texas Impact Evaluation Final Report. 


As total costs in the demonstration increased, there were also substantial changes 
in the distribution of costs among activities. Direct delivery costs decreased from 80 
percent of total costs before the demonstration to just 43 percent of total costs during 
the demonstration. Support services increased from 14 percent to about 50 percent of 
total costs. Among the various supportive services, the biggest change occurred for 
child care costs, which increased from 3 percent to 44 percent of total costs. The share 
of costs spent on administration did not change. The comparison site spent the bulk of 
its funds on direct service delivery and little on supportive services. Table E-6, which 
displays per participant costs for the major cost categories, makes it clear that the major 
difference in costs between the demonstration and comparison sites was the large 
amount spent on child care in the demonstration. 


Table E-6 
Costs per Participant in the Demonstration and Comparison Sites 
(Average for Two Demonstration Years) 


Demonstration Comparison 


Administration $69 $38 
Direct delivery $515 oad 
Support services $607 $38 
Total $1,191 $453 

Child care $520 N/A 


Source: Texas Cost Analysis Final Report, Figure 3.9, p. 20. 
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The difference in costs per participant hour between the demonstration and 
comparison sites was much smaller than the difference in costs per participant. For 
example, during the first year of the demonstration per hour costs were only 30 percent 
higher in the demonstration ($3.73 as compared to $4.31 in the comparison county) 
while per participant costs were 200 percent higher. In the second year of the 
demonstration, per hour costs were actually lower in the demonstration than in the 
comparison site ($4.31 as compared to $5.37) due to declining enrollments and the 
introduction of a new training component in the comparison site. These relatively small 
differences in hourly costs suggest that the added cost of the demonstration primarily 


supported longer activities. 


The Texas demonstration was able to access just over a million dollars in 
leveraged funds during the two-year demonstration project. These leveraged funds 
provided an additional $850 in resources per participant. Most of the leveraged funds 
came from Pell Grants, student loans, and personal earnings. The remainder came 


from adult education programs, the community college, and JTPA. 


SUMMARY 

The Texas demonstration tightened work registration requirements, encouraged 
volunteers, and offered a broader array of services to E&T participants. Key impacts 
of the demonstration include: 

e A small increase in the number of work registrants (about 8 percent) due 


primarily to eliminating the exemption for individuals caring for 
children aged 3 to 5. 


e An increase in the participation rate among mandatory registrants. 
e Substantial participation by volunteers. 


e Increased service to females, whites, individuals age 30 or under, and 
high school graduates, accompanied by reduced service to one-person 
households. 


e A large decline in the provision of directed job search offset by an 
increase in the provision of educational services. 


e An increase in the intensity of training as measured by total hours per 
month, primarily because of the large amount of educational services 


provided. 


e Little or no effect on employment and earnings outcomes. 
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e A large increase in costs, both aggregate and per participant. The 
increase in the cost per participant hour, however, was much smaller. 


e A large increase in leveraged funds from JTPA and educational sources. 
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Summary of Evaluation of the E&T/JOBS Conformance Demonstrations: Final Synthesis of 
Implementation and Process Evaluations, and Evaluation of the E&T/JOBS Conformance 
Demonstrations: Final Synthesis of Impact and Cost Evaluations 


From October 1, 1993 to September 30, 1996, the Food and Nutrition Service of the U.S. 
Department of Agriculture sponsored demonstration projects in Georgia, Hawaii, Missouri, 
South Dakota, and Texas to demonstrate the feasibility and effectiveness of operating the Food 
Stamp Employment and Training (E&T) program under the same legislative and regulatory 
terms as the Job Opportunities and Basic Skills (JOBS) program for Aid to Families with 
Dependent Children (AFDC) recipients. Common objectives of the demonstrations were to 
increase compliance with E&T participation requirements among mandatory work registrants, 
target services to individuals most at risk of long-term dependency and those most likely to 
benefit from E&T services, improve participant outcomes, and improve the cost efficiency of 
welfare to work services. 


States conformed their E&T programs to the JOBS model by changing their participation rules 
and E&T service designs. Each state was also required to design and arrange for an independent 
evaluation of its demonstration. The first volume of this report synthesizes the implementation 
and process evaluations of the demonstrations in the states. The second volume is a synthesis of 
the states’ impact and cost evaluations. 


The implementation and process evaluations focus primarily on the objectives of the 
demonstration and how the states designed their programs. Outcomes are covered in the impact 
and cost evaluations. These impact evaluations provide evidence of at most modest 
improvements in participant outcomes. Some of the state findings include: 


e Client targeting criteria and service approaches established for poor households with 
dependent children do not automatically make sense when transferred to the Food 
Stamp Program, since the demographics and participation dynamics of Food Stamp 
recipients can vary significantly from those of the AFDC population. 


e Intensive services were provided to a limited number of work registrants under the 
demonstration. It is unclear how the provision of these services would have affected 
outcomes for those work registrants who did not receive them, but probably would 
have under the regular Food Stamp E&T Program. 


e Selective targeting to priority groups can increase services to these groups. However, 
providing supportive services such as child care assistance to E&T participants may 
entice them to stay on the Food Stamp Program longer. 


The experiences from the demonstration states suggest that regulatory conformance and 
administrative consolidation between E&T and JOBS is administratively feasible at the state and 
local level. However, the findings described above may cast doubt as to whether such 
conformance improves outcomes for E&T participants. Moreover, the world of welfare to work 
has changed since these demonstrations were first authorized. Welfare reform replaced the 
AFDC program with state Temporary Assistance for Needy Families block grants and brought 
about significant changes to the Food Stamp E&T program. 
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